You have three choices when it comes to devices: (a) buy a new device, (b) buy a used/secondhand device, and (c) bring your own unlocked device.
Buying a New Device
If you decide to buy a new device, this would be done at the mobile phone service provider’s store. If you have a credit card（or a debit card can sometimes work too) you may be able to register it for installment payments on the device. If you don’t have a credit card, it is very likely you will be asked to pay for the device in full. Although it may be the most costly option, this option involves the least number of steps for setup.
Buying a Used/ Secondhand Device
With this option, you would buy a used or secondhand device from a separate retailer or third party (e.g., Amazon, some GEO stores, a fellow ALT/CIR) and take it in to the mobile phone service provider to activate service. It is very important to note that in the case of a SIM-locked device, you should buy one that is locked onto the network of the service provider you wish to use. If you choose this option, be sure to confirm with the service provider that they have a SIM card on hand that can be inserted into the device you chose; this can be done with a quick phone call (in Japanese, though).
Bringing an Unlocked Device
If you bring your unlocked device from home, make sure you can insert a SIM card into it. You will also need to check whether your phone will work on a Japanese network. You can work this out by using this website: www.willmyphonework.net