In this paper, a method for real-time tracking of moving objects or pedestrians is proposed. Especially, we tackled to track an object whose part was temporally occluded or which traversed in front of the cluttered background. Furthermore, we tried to obtain the accurate trajectory of the center of it stably even then. For the problems, we present a probabilistic color-based tracking. In our proposed method, we incorporated particle filtering into graphical models and applied it to the color-based tracking. When the color histogram of the tracked object was made, we used not one region of the whole of the object but the multi-part region of it if it was divided in some parts (e.g. the head and the body of a pedestrian). We treated the multi-part region as a graphical model. In the graphical model, messages about the state of the parts are sent to other parts. As the result, even when the tracked object has occluded part, our proposed method can track it stably and infer the state (e.g. position) of the occluded part. We made experiments to confirm effectiveness of this proposed method.