Talk2Nature
← All research notes

A MAP FOR BUILDING ON PRIOR WORK

Start from
what exists.

A useful library should change what we build next. Follow a question to the evidence, the available tools and an experiment worth checking.

Updated 2026-10-25 · Editorial recommendations, not validated integrations. Each linked note states exactly what was reviewed.

PATH 01

Choose the first animal

Where can we collect useful observations responsibly?

Welcome every animal in the app; validate one species and one observable question first.

Our proposed next step

Confirm access to several households with the same parrot species; compare annotation feasibility before committing to a pilot.

Budgerigars are provisional. No recruited participants, expert sign-off, recording uploads or translation model are implied.

PATH 02

Capture useful context

What should people record and label?

Reuse annotation workflows and independent behavior labels. Make unknowns explicit.

Our proposed next step

A synthetic file-exchange check, followed by a species-specific codebook and a reviewer-agreement exercise on approved data.

Real observations require an approved protocol, rights and consent. Public clips are not automatically research data.

PATH 03

Find sounds worth reviewing

How do we detect events without missing the quiet ones?

Compare existing detectors and encoders against our simple sound-level trigger. Keep a continuous, independently annotated test sample.

Our proposed next step

Specify a rights-cleared continuous-audio evaluation with quiet intervals, event recall, false alarms per hour and review time. Use MeerKAT for metric/schema guidance only unless a separately scoped non-commercial reproduction passes its rights and animal/session split review.

Triggered clips alone cannot establish recall. MeerKAT's non-commercial data and checkpoint terms exclude them from the shared commercial-capable path; its public sample names do not expose animal/session groups. HyenaSET's dataset is CC BY-NC-SA 4.0, and its individual-held-out split is useful only as a methods reference until group files and location sensitivity are reviewed. Its reported animal2vec baseline pretrains on all available audio, so treat it as transductive unless clarified. Specialized recorders do not establish phone performance.

PATH 04

Reuse models, then test transfer

Can sound help predict independently observed context?

Start with frozen audio features and a small classifier. Compare against context-only, background and simple acoustic baselines.

Our proposed next step

Choose a rights-cleared, independently labeled task; pin every data and model component; and freeze animal/session/site holdouts before evaluating simple baselines.

The dog-bark paper predicts labels tied to induced stimuli; Barkopedia's reviewed context labels were initially inferred from video and its source recordings' rights remain unresolved; CowRumIA reuses its test partition in validation callbacks. Treat each as a methods comparator, not evidence of meaning or a ready training set. No Talk2Nature animal model has been trained or benchmarked; AutoResearch remains NOT READY. iNatSounds is a weakly labeled file-level species-classification/pretraining reference, not a context or meaning set; its download terms restrict use to non-commercial research/education and prohibit audio redistribution, so it is not admitted to the commercial-capable Talk2Nature path. MooLogue is a mobile-learning comparator only: its paper and pinned repository disagree about offline versus cloud behavior, and code/data rights remain unresolved. BirdNET Live v1.2.1 identifies bird species, not call meaning. Its MIT code and Apache-2.0 model licenses are distinct; developer privacy claims and optional third-party location services were reviewed from docs, not runtime-tested.

PATH 05

Know what would count as understanding

When does an association support a claim about meaning?

Study receiver behavior, matched comparisons and competing explanations. Distinguish companion parrots' learned human names from natural conspecific signals proposed as names; keep each critique connected to the original evidence.

Our proposed next step

Ask a scientific reviewer to assess one bounded hypothesis and its controls before designing any intervention.

A caregiver report about a human-created name differs from evidence about natural conspecific calls. Neither an individual acoustic signature nor a receiver response alone establishes shared meaning. Public tools remain passive; no automated playback or general translation claim.

PATH 06

Connect sound, movement and other signals

Which measurements could reveal something audio alone misses?

Use separate evidence standards for animal behavior, plant physiology and fungal electrophysiology.

Our proposed next step

Start with an already-defined visual or acoustic signal and an independently measured outcome. Compare human review and simple baselines before proposing a phone feature; keep device feasibility and privacy gates explicit.

The honeybee camera study is a calibrated decoder for one known signal, not general gesture translation. Its linked code and ground-truth data have no located reuse license; no files were acquired. The phone-camera use case remains untested.

PATH 07

Design technology around the animal

What changes when an animal uses the interface directly?

Use species- and individual-aware interaction design, short optional sessions, and a reliable way to leave; test observable behavior rather than assuming engagement.

Our proposed next step

Compare animal-operated tablet tasks with parrot choice in video calls, then ask an animal-computer-interaction researcher to review any future animal-facing design before prototyping.

This evidence concerns touchscreen enrichment, not Talk2Nature’s human-operated passive recorder or a way to communicate meaning. This reading path does not add animal-directed features.

PATH 08

Align sound with observed behavior

Can an acoustic feature predict a behavior that was independently observed at the same time?

Treat overlap separation as an upstream preprocessing candidate, then evaluate caller attribution and independently observed context as separate tasks. Preserve unknown and mixed-source cases.

Our proposed next step

First establish rights and a source-resolved evaluation set. Compare known-source synthetic mixtures with and without separation, then test any downstream context claim on frozen animal/session groups with independent observation labels and appropriate baselines.

BioCPPNet reports validation on constructed mixtures and downstream caller identity; it does not validate natural phone recordings or receiver comprehension, and the reviewed code snapshot has no visible LICENSE. Bat-archive raw-audio rights and joins remain unresolved, and no source is admitted to the first Talk2Nature context benchmark.

PATH 09

Retrieve observable events without inventing meaning

Can a natural-language query find video clips whose reviewed behavior labels satisfy it?

Treat a prompt as a search interface over an explicit label schema. Evaluate perception, label provenance, prompt parsing and retrieval separately; do not infer vocal meaning from video labels.

Our proposed next step

Compare the two MammAlps releases' modalities, splits and licenses, then write a leakage-safe retrieval protocol with held-out individuals or sites before considering a model run.

Prompting-MammAlps is video-only and CC BY-NC 4.0; its code rights are unresolved and its published day split is not verified animal-held-out. Neither MammAlps release establishes animal language or enters the first commercial-capable benchmark.

Keep this map growing

Suggest an original paper, official code repository, dataset or correction. Include the question it helps answer, what was actually reviewed and the reuse limits. We prioritize methods that could change the next experiment; full methods review and specialist review remain separate steps.

Contribute a source or correction ↗