I tried messing around with it, but I couldn't get your current algorithm to work, so I tried it with a different one.
Lbl ROT
r1*C-(r3*S)//128->r5
r1*S+(R3*C)//128->r3
r5->r1
r1*48//r3+48->{r4}^^r
r2*48//r3+32->{r4}^^r
Return
If the sign bit of r3 is set, then the vertex was behind the camera and you shouldn't draw it. This code is just something that seemed to work for me, nothing more, so it's possible that it isn't the most optimized version and that there are still a few problems with it (especially since I programmed it at 2:00).
Keep in mind that a line of 10 units tall and only 3 units away from the camera requires a very wide field of view to be seen. I've tried a line of 10 units tall and 20 units away.

On irc, you also asked me what the 48 means at the start of the last 2 lines. This is a constant related to the screen's size and the FOV.
This is a sketch of the camera:

The angle between the black lines is the field of view (in this case 90°). The green line represents the screen, which is in our case 96 pixels wide. The blue distance, which is perpendicular to the screen, is the constant we need. With a field of view of 90°, this distance is equal to half of the screen width, which is 48.
I hope that this makes it clear. If not, feel free to ask and I'll try to improve my explanation tomorrow.