The more I think about it, the more sensible seems the idea of pumping heat out of extracted air into hot water tanks. Given a reasonably well sealed thermal envelope, the places you want to extract air from a house are kitchens, bathrooms and toilets. These are also places where hot water is used.
And if you don't have a well-sealed thermal envelope, then extracting air is not an issue.
If you extracted 50 cubic metres and dropped the temperature by 20 degrees, 1,300 kJ would be available. If you did this every hour, you'd get about one kWh every three hours, 8 kWh per day. According to Without Hot Air by David Kay, in Sustainability without the hot air, a bath takes about 5kWh and a shower 1.4 kWh. He estimates 12 kWh of hot water per day per person, although he seems to include cooking, refrigerating and freezing in his sums.
The problems, of course, are in economies of scale and system complexity.
In the summer, rather than cooling the air going out, you would want to cool the air coming in, but you probably wouldn't want to be drawing air into the house via the kitchen, bathroom and toilet!
Air conditioners are now pretty much standard fittings in Japanese houses and models are available that heat water as they cool the air, but these are not widespread, and in installation they work out more expensive than buying separate units for heating water and cooling air, and since the air conditioner is not on for most of the year, another means of water heating is necessary anyway.
Useful physical characteristics of air:
Air holds 1 kJ per kg per degree change in temperature.
In cubic metres, that's about 1.3 kJ per cubic metre kelvin.
