Tying the token to an IP address will largely ameliorate that issue. As always, there is a tradeoff of security and convenience. I've worked at facilities with no internet access, where hard drives were removed and put in safes at the end of the day, and that had "leper lights" which flashed when unsecure people like me were present. Very secure but hardly convenient. You always need to ask yourself what it is that you are securing.
You should not tie any functionality, user experience, and last but certainly not least, security to an IP address. IP addresses can be spoofed rather easily, using public wifi, hotel/guest networks, and even certain ISPs means the rotating of IP addresses on each user request.
Adding an IP lock to a token can only increase security - it cannot decrease security. And if you wish to be secure, then you wouldn't use public wifi.
What you're doing is increasing complexity, even if marginal, for no benefit and to the inconvenience of your users; Sometimes, increased complexity does introduce insecurities, but I digress. The developer has little to no control over what type of network (wi-fi, public hotspot, hotel, corporate, private, etc..) a user accesses the site with and the user couldn't care less how sessions are managed (for the most part), all they know is if their token becomes invalid on each request because they are using a network that pools and rotates IP addresses b/w users/requests, forcing them to re-authenticate with each request, well, you're going to have some very unhappy users.