There is no dress code for Japanese temples. If you are going to a special ceremony that seems formal, but most people treat them as a tourist site, it's normal to see visitors wearing jeans, shorts and so on.
You may be asked to remove your shoes if you want see the inner rooms. And in the inner rooms, it is not allowed to take photos even if permitted of the structure as a whole.
As many people come to these sites to pray, keep quiet especially when you approaching in front of the main Temple or inner rooms.
As conclusion, visitors are concerned about the dress code, but you don't need to concern the dress code for Japanese temple.