Pieces of the Puzzle
Connecting to a Linux machine via Bluetooth works now. Data transfer seems to work as well, but that's not really verified. The 3650 and Mac OS X don't like some of the connection parameters used in the top level RFCOMM layer, but much of the lower layers is implemented. Some mysteries still need to be solved, for example the timing when sending packets via the HCI layer. Sometimes, the packets get lost... what would help is using delay timers, but that's an area I've tried to avoid so far. Timer events should however be a universal feature on the NewtonOS, and they would help in making the Bluetooth connection more reliable.
I also cracked open a no-name USB Bluetooth dongle to see what kind of module is inside. Unfortunately, Bluetooth modules often come in either UART or USB flavor, and the module I found is a USB only module. So much for getting cheap internal Bluetooth on the Newton! The modules from BTDesigner.com are not that expensive, but shipping is about $30...
One thing that got a bit in the way of my development work is that Mac OS X 10.2.8 broke my Bluetooth setup. The system preferences crash in the Bluetooth panel (yes, I've deleted all relevant .plist files) and my Acer dongle doesn't work anymore. Seems like Apple wants people to upgrade to Panther.