doesn't 4 batteries at 1.5 volts each make 6 total?
anyways, i tried this before as well, on a whim, but decided against using it after that for fear of making something explode.

I've had 8.1 volts going to it before at about 1.5 amps and it worked fine. I did this to the gameboy as well except I used 9 volts and the amps may have been slightly higher. I know on the gameboy, after a certain amount of voltage, it just refused to work. I don't think you will harm the calc much, but I would not suggest over 9.6 volts (8 rechargeable batteries).

You trying to bring it to life Dr. Frankenstein style?
Brute force code breaking?
Portals?

For the record, the TI-84+/SE's maximum current draw without USB is about 30 mA, which happens during flash write/erase operations; the typical current draw is 16 mA max in other circumstances, 5-12 mA typical, and 2 mA on the homescreen with no hooks. For comparison, when a USB device is initially connected, it can draw up to 100 mA, and up to 500 mA if authorized by the computer. The USB port on the TI-84+SE is a USB-On-the-Go port, and supplies a maximum of 100 mA to a connected peripheral.

The TI-83+/SE/84+/SE family of calculators test the batteries by reading the voltage with the CPU in a busy-loop (which raises the current draw). The TI-83+ was originally based on 5.0 V logic, and the CPU specifically tests for voltages around this level. The newer TI-83+s and the SE/84+ series of calculators are based entirely around 3.3 V logic, but the OS still expects at least 5.0 V, even though the hardware (except for the USB port +5 V power rail) technically does not need it.

The calculators use linear voltage regulators, which convert the extra voltage supplied by the batteries into waste heat. Consequently, the calculators are fairly flexible with respect to the voltages they can accept. The maximum voltage you can supply to your calculator is dependent upon the load and design of the regulators TI uses. We experimentally know they will accept up to 9 V; the maximum is probably between 12 to 15 volts. Because the OS specifically looks for around 5.0 V, you should not supply less than 5.0 V. Due to various issues like voltage losses in the USB cable, a USB port might actually supply slightly less than 5.0 V. Alkaline batteries supply 1.5 V for between 250 and 1200 mAh; rechargeable batteries typically supply 1.2 V, with a similar capacity range, though usually lower. Because rechargeables supply a lower voltage, they can easily fail to meet the OS's voltage expectations.

Generally, it is perfectly acceptable to run a calculator from a generic power supply. However, a poor-quality power supply may permit excess AC noise to reach the calculator, causing stability issues. Adding a capacitor may help filter out noise. For a USB port, a capacitor may stabilize the voltage output, as changes in the calculator's load can cause transient voltage drops with a poor-quality USB power supply.

Edit: I created an official power-related Wiki page.

The only good option for using rechargeables in your calculator are rechargable alkaline batteries. The only other option I can think of would be lithium-based batteries, which operate at around 3 V, and so would either require using specially shaped batteries or dummies.