Playing with toy blocks reveals patterns in children’s play that are valuable for therapy and assessment. Following the 2011 Tohoku Earthquake and Tsunami in Japan, we witnessed young survivors expressing post-trauma stress in block play. Motivated by the limitations in assessing this behavior using traditional methods, our paper describes the design rationales of AssessBlocks, an action-characterizing system using smartwatch embedded toy blocks. Utilizing a smartwatch’s Inertial Measurement Unit (IMU) and capacitive screen, a monitor is able to receive, visualize and document sequential and quantitative play actions that were empirically selected from a preliminary block therapy study on children’s post-disaster stress. We also propose our vision of a multi-dimensional behavioral assessment system using actions obtained by AssessBlocks.