So apparently, when using Curve to Points node, the rotation isn’t aligned exactly to the segment, but bends around corners for some reason. This is how it looks on poly spline:
Things can be improved if you convert to Bezier spline and scale the point handles to 0, but points very close to the corner still get misaligned
Is there a better way to align instances to curve segments or is this a bug?