Heat exhaust from a wing surface is a key technology in developing a high altitude long endurance unmanned aerial vehicle (HALE UAV) that satisfies the requirements of both aerodynamics and propulsion. This paper identifies the exhaust heat characteristics depending on the airfoil shape at low Reynolds number to design a HALE UAV with a wing surface heat exchanger. In the first step, computational fluid dynamics (CFD) simulations are conducted for three different airfoil shapes at low Reynolds number. Each airfoil is investigated by considering the heat exhaust from different regions on the upper wing surface at different angles of attack. As a result, two significant characteristics are revealed. First, a turbulent boundary layer promotes heat transport by mixing a thermal boundary layer. Thus, early transition near the leading edge is favorable to improve heat exhaust characteristics while it may increase skin friction. These results indicate a trade-off between aerodynamic performance and heat exhaust performance. Second, heat exhaust should be conducted only in the region of turbulent boundary layer behind the transition point rather than in the whole region including both laminar and turbulent boundary layers. From these results, the airfoil shape significantly affects the Nusselt number distribution along the upper wing surface due to the change in the location of laminar-turbulent transition and turbulent boundary layer separation. In the next step, the multi-objective aerodynamic optimization of an airfoil shape is carried out at low Reynolds number and the optimized airfoils are compared in terms of their heat exhaust characteristics. The optimized airfoils have qualitatively same trends of the Nusselt number distributions as the airfoils tested in the parametric study. However, there is a difference in the Nusselt number distribution within the turbulent boundary layer region among these optimized airfoils. When we design a HALE UAV with a wing surface heat exchanger, we should consider not only the location of laminar-turbulent transition but also the entire flow field.