It's not legally permissible to apply for a work visa from within Japan while there on the 90-day tourist visa. I've heard of exceptions to that law, but I don't know the specifics. Normally, one has to apply for a longer (than 90 day) visa from one's country of residence (via the Japanese embassy or consulate). So for work visas, you have to already have something lined up to fulfill the requirement for an employer sponsorship. Similarly, a student visa would require prior acceptance at a Japanese school (for example, a Japanese language school, a route many people take). You can also apply for a Business Manager or Business Start-Up visa (my personal route was the former: I bought a business). And there are other visa pathways...

It would be very difficult to get hired without a proper visa and residence card (zairyu)...and anyone who would be willing to break the law like that would possibly be pretty sketchy to work for! There have been many instances of foreign agricultural workers, light manufacturing workers, garment workers, etc. (usually Southeast Asians) getting screwed over by unscrupulous employers.

In more rural, agricultural areas, there is a big need for workers...but if you don't have at least basic conversational Japanese, things would be very difficult: Japanese people outside of the biggest cities seldom speak more than VERY basic English. It would also be extremely difficult to rent a place to live w/o proper documents (and some Japanese language ability). Honestly, it can be a real challenge for even perfectly legal foreigners to rent places in Japan...as lot of people simply won't rent to gaijin.

And yes, if you get caught by the cops without documentation of legal residence, you will be arrested and eventually deported. Japan is pretty hardcore about immigration enforcement.

Interestingly, though, while Japan has a low number of homeless people, as long as they don't make a mess (and they don't...), people are pretty kind to them and the cops basically ignore them. Sleeping "rough" isn't really a problem.