feat: resume recording into an existing dataset across sessions#74
Open
ZouzouWP wants to merge 1 commit into
Open
feat: resume recording into an existing dataset across sessions#74ZouzouWP wants to merge 1 commit into
ZouzouWP wants to merge 1 commit into
Conversation
Recording a full dataset rarely happens in one sitting. Before this, adding more episodes to an existing dataset later meant starting a brand new one and merging them by hand — nothing stopped you from silently mixing sessions with different FPS, cameras, or task wording into a broken combined dataset.
## What changed
- `DatasetPicker` lists existing local/Hub datasets with their episode count, and offers a "Resume" action next to "Create new".
- Resuming pre-fills FPS, cameras, robot type, and the last task description from the dataset's own metadata (via a new `tasks` field on `/dataset-info`), and **blocks recording** if the current camera/robot setup doesn't match what the dataset was originally recorded with — this is exactly the mismatch that used to fail silently.
- An optional episode-count target (stored in `localStorage`, not the dataset itself) drives a progress bar across sessions, e.g. "14 / 45 episodes".
- Backend fix: `LeRobotDataset.resume()` was being called with `root=None`, which current `lerobot` rejects. Now resolves the same default local directory `LeRobotDataset.create()` uses.
```mermaid
sequenceDiagram
participant U as User
participant L as Landing page
participant API as LeLab backend
participant DS as LeRobotDataset
U->>L: Click "Resume" on a dataset
L->>API: POST /dataset-info {repo_id}
API->>DS: LeRobotDataset(repo_id)
DS-->>API: fps, cameras, robot_type, tasks
API-->>L: pre-fill form
Note over L: Blocks start if current setup mismatches
U->>L: Confirm, start recording
L->>API: POST /start-recording {resume: true, repo_id}
API->>DS: LeRobotDataset.resume(repo_id, root=resolved_root)
DS-->>API: existing episodes preserved
API-->>L: new episodes appended after the last one
```
## Testing
Validated end-to-end on the physical SO-101: episodes appended to a 5-episode dataset across two separate sessions on different days, episode count and video/frame continuity confirmed on the Hub afterward.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recording a full dataset rarely happens in one sitting. Before this, adding more episodes to an existing dataset later meant starting a brand new one and merging them by hand — nothing stopped you from silently mixing sessions with different FPS, cameras, or task wording into a broken combined dataset.
What changed
DatasetPickerlists existing local/Hub datasets with their episode count, and offers a "Resume" action next to "Create new".tasksfield on/dataset-info), and blocks recording if the current camera/robot setup doesn't match what the dataset was originally recorded with — this is exactly the mismatch that used to fail silently.localStorage, not the dataset itself) drives a progress bar across sessions, e.g. "14 / 45 episodes".LeRobotDataset.resume()was being called withroot=None, which currentlerobotrejects. Now resolves the same default local directoryLeRobotDataset.create()uses.sequenceDiagram participant U as User participant L as Landing page participant API as LeLab backend participant DS as LeRobotDataset U->>L: Click "Resume" on a dataset L->>API: POST /dataset-info {repo_id} API->>DS: LeRobotDataset(repo_id) DS-->>API: fps, cameras, robot_type, tasks API-->>L: pre-fill form Note over L: Blocks start if current setup mismatches U->>L: Confirm, start recording L->>API: POST /start-recording {resume: true, repo_id} API->>DS: LeRobotDataset.resume(repo_id, root=resolved_root) DS-->>API: existing episodes preserved API-->>L: new episodes appended after the last oneTesting
Validated end-to-end on the physical SO-101: episodes appended to a 5-episode dataset across two separate sessions on different days, episode count and video/frame continuity confirmed on the Hub afterward.