I have never fractured one but if the PT says stay off it then I'd listen to him/her. Which bone is it? The body has very few sesamoid bones but they all do about the same thing. Because they act a little like a pulley/lever they do see some reasonably high stresses so I would error on the side of over recovery. I doubt that it means you can't keep doing Tri's but you might need to bad your next race or two depending on the severity of the fracture, your age, which bone, how much rest you are getting (really), bla bla bla...