next up previous
Next: Finite Representation of Value Up: POMDPs and Value Iteration Previous: Value Iteration

Technical and Notational Considerations

For convenience, we view functions over the state space vectors of size tex2html_wrap_inline1136 . We use lower case Greek letters tex2html_wrap_inline1138 and tex2html_wrap_inline1140 to refer to vectors and script letters tex2html_wrap_inline1142 and tex2html_wrap_inline1144 to refer to sets of vectors. In contrast, the upper case letters V and U always refer to value functions, that is functions over the belief space tex2html_wrap_inline1036 . Note that a belief state is a function over the state space and hence can be viewed as a vector.

A set tex2html_wrap_inline1142 of vectors induces a value function as follows:

displaymath1134

where tex2html_wrap_inline1154 is the inner product of tex2html_wrap_inline1138 and b, that is tex2html_wrap_inline1160 . For convenience, we shall abuse notation and use tex2html_wrap_inline1142 to denote both a set of vectors and the value function induced by the set. Under this convention, the quantity f(b) can be written as tex2html_wrap_inline1166 .

A vector in a set is extraneous if its removal does not affect the function that the set induces. It is useful otherwise. A set of vectors is parsimonious if it contains no extraneous vectors.

Given a set tex2html_wrap_inline1142 and a vector tex2html_wrap_inline1138 in tex2html_wrap_inline1142 , define the open witness region tex2html_wrap_inline1174 and closed witness region tex2html_wrap_inline1176 of tex2html_wrap_inline1138 w.r.t tex2html_wrap_inline1142 to be regions of the belief space tex2html_wrap_inline1036 respectively given by

eqnarray162

In the literature, a belief state in the open witness region tex2html_wrap_inline1174 is usually called a witness point for tex2html_wrap_inline1138 since it testifies to the fact that tex2html_wrap_inline1138 is useful. In this paper, we shall call a belief state in the closed witness region tex2html_wrap_inline1176 a witness point for tex2html_wrap_inline1138 .

   figure178
Figure 1: Illustration of Technical Concepts.

Figure 1 diagrammatically illustrates the aforementioned concepts. The line at the bottom depicts the belief space of a POMDP with two states. The point at the left end represents the probability distribution that concentrates all its masses on one of the states, while the point at the right end represents the one that concentrates all its masses on the other state. There are four vectors tex2html_wrap_inline1194 , tex2html_wrap_inline1196 , tex2html_wrap_inline1198 , and tex2html_wrap_inline1200 . The four slanting lines represent the linear functions tex2html_wrap_inline1202 (i=1, 2, 3, 4) of b. The value function induced by the four vectors is represented by the three bold line segments at the top. Vector tex2html_wrap_inline1198 is extraneous as its removal does not affect the induced function. All the other vectors are useful. The first segment of the line at the bottom is the witness region of tex2html_wrap_inline1194 , the second segment is that of tex2html_wrap_inline1196 , and the last segment is that of tex2html_wrap_inline1200 .


next up previous
Next: Finite Representation of Value Up: POMDPs and Value Iteration Previous: Value Iteration

Dr. Lian Wen Zhang
Thu Feb 15 14:47:09 HKT 2001