'tis indeed confusing. If I said I'd lost my purse, I'd mean the small, leather, money wallet-type thing with a clasp - the one I keep my cash in. I wouldn't mean that I'd lost my handbag or my shoulder bag.
A tote bag to me, means a bag with long handles or something that used to be known as a 'duffle bag' (back in the mists of time) or a bag you can just get on your shoulder and have the rim of the thing under your arm.
I still find it odd that Americans say 'purse' for hand or shoulder bag and 'pocket book' for purse.
As the man said: 'Two countries divided by a common language.'