Domestic dog (Canis familiaris), Siberian Husky and Shiba Inu / Dataset and audio-classification study

EmotionalCanines: video-derived labels are not a dog-reported emotion

Publication year 2025 · Note reviewed 2026-10-14

The paper predicts two three-class labels from audio: arousal (low/medium/high) and valence (negative/neutral/positive). Annotators first watched the associated video with surrounding context and used visible behavior and situation to assign the labels; samples were retained only when at least two of three raters agreed on both dimensions. These are human video-derived behavior/affect annotations, not an animal-reported state, an independently validated physiological measure, a receiver response, or evidence that a bark means a particular thing. The article reports dog-separated tests with best accuracies around 50–57% across breed and target combinations. It separately reports higher 10-fold cross-validation results, but the reviewed methods do not specify whether those folds were grouped by dog. The Hugging Face path formerly used for Barkopedia Task 6 was rewritten at the pinned revision as EmotionalCanines and its license metadata changed; no continuity of the underlying files was verified.

What we reviewed · Targeted full-text methods/results review; version-pinned dataset metadata review; no independent reproduction

Read the author-hosted eight-page ACM Multimedia paper sections on dataset generation, annotation, reliability, dog-separated splits, baselines, results and limitations. Reviewed the exact Hugging Face card commit c0617b15562882b9eee4cc1bcb8e6aa80e51bd34 and its README diff, plus the pinned GitHub README/LICENSE at 6b73c5fce55fc132e97db6a76899f8594a955f59. The paper reports 1,400 bark sequences (700 per breed; 600 train and 100 test per breed) from 199 dogs, with dog identities separated between train and test. Twelve annotators contributed; each sequence received three labels, and a 100-sample pilot reported Fleiss kappa 0.30 for arousal and 0.42 for valence. No recordings, labels, code or model weights were downloaded or run; no metrics were reproduced.

No independent animal-affect, canine-behavior or bioacoustics review has been received.

Study at a glance

Question and target: Audio classifiers predict human-assigned three-class arousal and valence labels; they do not test whether a receiver understood a call.
Dataset and split: Paper reports 1,400 sequences, 35:02 total, two breeds, 199 dogs, 600 training and 100 test sequences per breed, with no dog overlap between train and test.
Label procedure: Twelve annotators; three raters per sample; a 100-sample pilot reported Fleiss kappa 0.30 (arousal) and 0.42 (valence); retained samples required majority agreement.
Evaluation: Authors report dog-separated test accuracies around 50–57% for the best breed/target combinations. Higher 10-fold CV results are reported separately; dog grouping of those folds was not established in the reviewed methods.
Release lineage: The former Barkopedia Task 6 Hugging Face README was rewritten as EmotionalCanines at c0617b1 and now points to a separate GitHub release; file continuity was not checked.

Reading location: Author-hosted PDF, pages 1–5: dataset construction, annotation, agreement, splits, baselines, reported test results and conclusion; Hugging Face README diff at revision c0617b15562882b9eee4cc1bcb8e6aa80e51bd34; GitHub README/LICENSE at commit 6b73c5fce55fc132e97db6a76899f8594a955f59.

What this does — and doesn’t — establish

The paper says its source pipeline began from internet video-derived bark sequences and acknowledges engagement-related sampling bias. The emotion labels depend on human interpretation of video behavior/context; the 100-sample pilot reports fair agreement, not external ground truth, and the authors discuss the difficulty of directly measuring subjective state from visible behavior. The dataset covers two breeds and only 35 minutes of bark sequences; each breed has 100 held-out test sequences. The reported 10-fold CV grouping is not clear from the reviewed methods, and no baseline code or results were independently reproduced. The dataset repositories declare CC BY-NC-SA 4.0, but permissions, consent and privacy terms for original source videos were not independently traced. Do not treat audio classification as emotion diagnosis, welfare advice or communication translation.

What we can build on

For Talk2Nature, store a person’s directly observed behavior separately from model output and from any inferred affect. If an affect question is ever studied, predefine the behavioral/physiological evidence and independent validation; avoid presenting a video-derived human label as the animal’s inner state.

A practical starting point

Metadata-only methods reference; not an admitted Talk2Nature training dataset, emotion detector or communication benchmark.

Available starting point: Author-hosted article, pinned Hugging Face card revision c0617b15562882b9eee4cc1bcb8e6aa80e51bd34, and pinned GitHub dataset repository commit 6b73c5fce55fc132e97db6a76899f8594a955f59. No files were acquired.

Reuse limits: The article and dataset separately declare CC BY-NC-SA 4.0. The dataset terms do not establish rights or consent for original internet videos; this non-commercial dataset is not admitted to the shared commercial-capable collection.

Our proposed experiment: No experiment was run. Any future test must distinguish audio association with observer labels from animal emotion or meaning, and must establish the exact media rights, a qualified review, independent reference labels and grouped evaluation.

The next reading task

Before considering any dataset use, resolve the original video/audio permissions, consent and privacy basis, identify whether the current GitHub and historical Hugging Face files are the same release, and obtain specialist review of the annotation construct. A future benchmark would need a frozen dog/session-grouped test and independent behavioral or physiological reference; the current record remains metadata-only.

Go to the sources

EmotionalCanines: ACM Multimedia 2025 paper (author-hosted full text) ↗
Eight-page paper describing a video-annotated dog-bark arousal/valence dataset and audio-classification baselines.
EmotionalCanines dataset card and Barkopedia Task 6 card revision ↗
Hugging Face commit c0617b15562882b9eee4cc1bcb8e6aa80e51bd34 rewrites the former Barkopedia emotion-task card as EmotionalCanines.
EmotionalCanines dataset repository and license at a pinned GitHub commit ↗
Author repository for the dataset linked by the current Hugging Face card, pinned to commit 6b73c5fce55fc132e97db6a76899f8594a955f59.

Original editorial note by Talk2Nature contributors. AI-assisted, source-grounded synthesis; not a peer review or an independent replication. Linked works retain their own rights.