Ask them individually, starting with the kids. That way you don't rat yourself out to your wife if it wasn't her. If they deny it, look for any sign of embarrassment. Obviously keep a cool head, whoever it was, just explain it or at least treat them with an open mind.
Otherwise, let it go and...