So here we go.
Handmade adapter for dealing with LTC1799's package size.
If you can't or just don't want to make your own adapter check out Getlofi.com for a LTC1799 kit.
Now the cap I used (image below) was not ceramic or electrolytic. Honestly I do not remember the type I got it awhile ago. I'd guess it is mylar and not tantalum.
And again if your toy uses 4.5V (3 1.5V batteries) you should be fine without the 5 volt regulator.
When you tap into the toy's power try to find a point on the board that only has power when the toy is on, otherwise put in a "kill/reset switch" that will actually turn off everything (toy and 1799 circuit) to not run down batteries.
The OUT is what you can probe the toy with (try this for kicks before removing any components) just stay clear of power components.
Below is a pinout for the main chip in V-tech Alphabet Desk.
Notice pins 7 & 8, and then refer to the diagram below the pinout.
This circuit shows how the xtal, coil, or resonator would be hooked up. Inside the chip is inside the thicker U-shape. So if you remove the external components you can hookup your own osc. Now xtal, coil, or resonator are not "directional", one side IN other side OUT. What does have a direction is the NAND gate in the chip. INIT and OSC1 are both inputs. CLK and OSC2 are the same line and the Output. The Output feeds back through the component of choice. That with the 2 caps is what sets the frequency of the osc. The 1M res connected between Output and Input makes the NAND act like a 10x amplifier. INIT is held HIGH so that when OSC1 is HIGH Output is LOW and when OSC1 is LOW Output is High, hence oscillating HIGH-LOW. If INIT is brought LOW the entire circuit resets (got that bit of info from datasheet).
But so now back to where to attach your osc circuit. You can probe both OSC1 and 2 and your signal will be similar, but one of the two should be a little better and a bit more stable.
One way I have probed (before removing any components) is to use an LED with 1k res. GND >> res >> LED >> OSC1 (and then OSC2). One pin will make the LED brighter and that's the one you want (after removing components).