Silica dessicant is not poisonous.
You still shouldn't eat it though, it's extremely absorbent and cause bowel obstructions by drying out part of your guts and getting stuff stuck.