Keypoint Annotation for Electrocommunication Source Separation with PIKAChU and RAIChU

Source separation is a key step in understanding animal communication, particularly in acoustic and electric modalities. Often, positional information is used to determine which animal emitted a given signal. Automatic keypoint tracking algorithms can provide this information, but they sometimes require a prohibitively large number of hand-labeled examples. We introduce a methodology for automatically labeling keypoints in weakly electric fish, which sense and communicate by emitting electric pulses. Specifically, we invert a physics-based electrogeneration model to reconstruct the position of the fish which emitted a recorded pulse. This approach allows us to transform inexpensive electric-signal recordings into pose estimates. Our algorithm makes feasible the study of social interactions and communication in schools of weakly electric fish by increasing the efficacy of source separation in electrocommunication settings. More broadly, we provide a general framework for augmenting pose estimation training data to improve source separation, with potential applications in acoustic modalities and beyond.