You're right it is very stressful and difficult. I've thought of walking away many times since in my opinion dealing with dishonest people is not worth it. I haven't signed anything ever with them. It's all been on good faith. I've been doing work on their servers and checking it into their repository I'm not sure what that means. Up here in the north east where there isn't a large pool of investors / firms like silicon valley I feel pressure to comply so this investor doesn't vanish.
Are you worried about investors? This is the time to take your slice of the pie(I mean in honest sense). If investors find out you own the code and can walk away with it they will be hesitant in investing since once you walk away what are the founders left with? they'll lose investor.
Also if you think they are not honest trust me it is not worth trying more, down the road they'll try something else to get back the stocks even if they give it to you now.
If they are not honest with you take your code without telling them and start you own company or if you need a job find other one. It is very difficult to get the dishonest people straight, best thing is to find your way out.
if you didn't sign anything at all, you might want to talk to your lawyer about IP laws and also talk to this investor in a one-on-one meeting, talking about whats happening.
just because they have and are using your code might not necessarily mean that they can continue to do so without your permission, if you didn't sign anything.