Making invisible work legible.

Abstract

CookCredit builds tools for the people who prepare food and keep service moving. Much of that work is visible in its outcome but absent from the record: the repeated practice behind a controlled cut, the preparation before service, and the coordination that keeps a kitchen functioning. Our purpose is to make useful parts of this work observable, explainable, and available for learning and human review.

Knife work is our starting point. EgoCut documents a method for following the relationship between a hand and a blade in ordinary cooking video, then identifying movement events that can support feedback. This statement describes that research direction, how it connects to CookCredit’s products, and what the measurements cannot establish.

About the company

CookCredit develops practical technology for food work: camera-based feedback for learning, recorded work samples for hiring, and tools for everyday restaurant service. The company’s starting point is the person doing the work. A useful system should explain what it observed, help that person or their reviewer understand it, and leave room for context.

We start with knife skills because they bring together timing, repeated movement, tool control, and a visible task. They offer a concrete setting in which to study how computer vision can support practice. The longer-term purpose is broader: to build a useful record of work that is often recognized only after it is missing.

1. The problem: work without a useful record

A finished plate does not describe the effort that produced it. A résumé cannot show the timing of a person’s hands. A count of completed tasks can miss preparation, cleanup, interruptions, and the support one person gives another. We use invisible labor to describe work that is necessary but poorly represented in the information used to discuss progress, contribution, or skill.

Making that labor visible is not the same as reducing a person to a score. A useful record should let someone inspect the evidence, understand the measurement, and challenge the interpretation. CookCredit connects camera-based practice, practical hiring review, and service tools through that principle.

2. Method: from video to movement evidence

The public EgoCut method begins with a single RGB video stream. It describes hand-pose extraction with MediaPipe and intermittent knife detections from YOLO. The technical idea is to estimate the blade’s position relative to wrist landmarks, rather than require the knife detector to succeed in every frame. Orientation is smoothed around the wrist; when the knife is obscured, its position is a prediction, not a directly observed fact. [1]

Candidate cutting events are evaluated using three complementary signals: depth relative to recent motion, movement after a potential impact, and the timing of repeated strokes. These gates aim to distinguish cutting from repositioning and isolated movement. Their output is a sequence of events and associated measurements, rather than a judgment about the person performing them. [1]

Wrist anchoring provides a reference that moves with the cook’s hand. A visible blade detection supplies an estimated offset from the wrist to the blade tip. Between detections, the hand trajectory carries that estimate forward. When another blade observation is available, it can update the estimate. This connects intermittent object detection to a more continuous movement trace, while retaining the distinction between an observed blade and an inferred one. [1]

The architecture separates trajectory handling, blade localization, stroke detection, and downstream consumers. The public repository describes a two-pass offline workflow and a target single-pass deployment. It exposes interfaces and documentation; the complete reference implementation, trained weights, and labeled benchmarks are not all publicly released. [2] [3]

3. Calculating what can be observed

For a recorded task, useful quantities include event count, time between events, variation across movements, and the portions of a recording where the work can be observed reliably. These are descriptions of a particular session under particular recording conditions. They are not universal measures of ability.

From stroke events to interpretable feedback

A detected stroke is a timestamped event. The gaps between successive events describe cadence; the variation in those gaps describes how regular the detected sequence is. Movement amplitude and trajectory features add context about how the action was performed. Looking at these quantities together helps separate a change in pace from a change in repeatability.

For example, two recordings can contain the same number of detected strokes but show different patterns: one may contain steady repetitions, while the other contains a burst followed by a pause. The count alone conceals that difference. A timeline and the corresponding video let a learner or reviewer examine the pattern and decide whether the pause reflects repositioning, preparation, or something the camera could not capture.

Accounting for invisible labor

Our broader question is how such evidence can help account for labor that ordinary workflow records overlook. That requires more than detecting movement. Task definitions, preparation and recovery time, off-camera work, context, and the worker’s own account all matter. An unobserved interval must remain unknown; it must not be silently classified as inactivity.

We do not claim that a stroke counter calculates the economic value of a person’s work, their productivity across a shift, or a fair wage. Connecting observed actions to a wider account of labor is a research direction, with validation and participation still required.

4. Evaluation and limits

EgoCut’s evaluation protocol calls for manual stroke annotation, cross-checks against audio impacts, and board-region annotation. It proposes stroke precision and recall, burst-level F1, and detection latency, reported separately by camera angle and frame rate. Those are evaluation criteria, not published proof that every capture condition performs equally. [4]

Camera motion, occlusion, lighting, technique, ingredient, and frame rate can change the observations. Image-space distances and angles are not calibrated physical measurements. Predicted landmarks can drift. A complete evaluation needs representative recordings and labeled reference events; this page does not present an accuracy benchmark.

Evaluation must also examine disagreements. A useful error analysis distinguishes a missed cutting event from a movement incorrectly counted as a cut, and asks whether errors cluster around occlusion, a particular camera position, or a change in task. Comparing the detected timeline with annotated video makes these failures inspectable and helps identify which part of the measurement chain needs improvement.

The research documentation and the published product are related artifacts with their own release histories. A method description should not be read as a guarantee that every documented feature is present in every live assessment.

5. Putting the mechanism to work

Learn & Practice gives people a way to examine a session and choose what to work on next. CookCredit for hiring brings a practical work sample and its recording into a conversation about a role. The employer makes the hiring decision; CookCredit does not automatically decide who should be hired.

For a learner, the practical question might be whether a repeated sequence became more regular under comparable conditions. For an educator, it might be which segment of a recording is worth discussing with a student. For a hiring manager, it is how a work sample relates to a particular kitchen role. The same observation can support each conversation, but the interpretation depends on the purpose and the people involved.

Toque addresses the surrounding service workflow: making the next action accessible at a table, station, tool, or supply. These products serve different tasks, but share an aim: make everyday food work easier to understand and carry out.

6. Programs and support

CookCredit participates in NVIDIA Inception and the Google for Startups Cloud Program. These programs provide access to technical resources and cloud support. Participation does not imply investment, endorsement, or independent validation of the research.