A new type of solar neutron detector (SEDA-FIB) was launched on board the Space Shuttle Endeavor on July 16 2009, and began collecting data at the International Space Station (ISS) on August 25 2009. This paper summarizes four years of observations with the solar neutron detector SEDA-FIB (Space Environment Data Acquisition using the FIBer detector). The solar neutron detector FIB can determine both the energy and arrival direction of solar neutrons. In this paper, we first present the angular distribution of neutron induced protons obtained in Monte Carlo simulations. The results are compared with the experimental results. Then we provide the angular distribution of background neutrons during one full orbit of the ISS (90 minutes). Next, the angular distribution of neutrons during the flare onset time from 20:02 to 20:10 UT on March 7 2011 is presented. It is compared with the distribution when a solar flare is not occurring. Observed solar neutrons possibly originated from the M-class solar flares that occurred on March 7 (M3.7), June 7 (M2.5), September 24 (M3.0) (weak signal) and November 3 (X1.9) of 2011 and January 23 of 2012 (M8.7). This marked the first time that neutrons have been observed from M-class solar flares. A possible interpretation of the neutron production process will be also provided.