We compute the twist-three fragmentation contribution to the transverse single spin asymmetry (SSA) in light hadron production p↑p→hX and p↑A→hX including the gluon saturation effect in the unpolarized nucleon/nucleus. Together with the results in our previous paper, this completes the full evaluation of the SSA in this process in the "hybrid" formalism. We argue that the dependence of SSAs on the atomic mass number in the forward region can elucidate the relative importance of the soft gluon pole contribution from the twist-three quark-gluon-quark correlation in the polarized nucleon and the twist-three fragmentation contribution from the final state hadron.