Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,12 +105,12 @@ static Future<bool> restartActiveDeviceMonitoring() // No-op on iOS

// Streaming
static Future<int> startStreamSession(
String? deviceUUID, {
String? deviceId, { // WearableDevice.id (getDevices) pins a pair; null = auto-select
double fps = 30.0,
StreamQuality streamQuality = StreamQuality.high,
VideoCodec videoCodec = VideoCodec.raw,
}) // Returns textureId for Texture widget
static Future<bool> stopStreamSession(String? deviceUUID)
}) // Returns textureId. Throws PlatformException 'STREAM_ACTIVE' if a different device is already streaming
static Future<bool> stopStreamSession(String? deviceId)
static Stream<StreamSessionState> streamSessionStateStream()
static Stream<StreamSessionError> streamSessionErrorStream()
static Stream<VideoStreamSize> videoStreamSizeStream()
Expand All @@ -134,7 +134,7 @@ static Future<bool> openDATGlassesAppUpdate()

// Photo capture
static Future<CapturedPhoto> capturePhoto(
String? deviceUUID, {
String? deviceId, {
PhotoCaptureFormat format = PhotoCaptureFormat.jpeg,
})

Expand Down Expand Up @@ -469,8 +469,8 @@ Develop and test without physical Meta glasses. Mock support lives in the option
```yaml
# pubspec.yaml — add only in dev/staging configs
dependencies:
flutter_meta_wearables_dat: ^0.5.2
flutter_meta_wearables_dat_mock_device: ^0.5.2
flutter_meta_wearables_dat: ^0.6.0
flutter_meta_wearables_dat_mock_device: ^0.6.0
```

```dart
Expand Down
5 changes: 3 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
## 0.6.0
* New `getDevices()` returning `List<WearableDevice>` — enumerate the paired glasses with their `name`, model (`WearableDeviceType`), link state, compatibility, and display capability. `isStreamingDevice` flags the pair the current stream is actually using; `isActive` flags the pair the shared auto-selector would bind a *new* stream to (the two differ when more than one pair is connected). Use it to tell two pairs of the same model apart and see which one you're streaming from. Read-only — *selecting* (pinning) a specific pair is planned for a follow-up.
* New `getDevices()` returning `List<WearableDevice>` — enumerate the paired glasses with their `name`, model (`WearableDeviceType`), link state, compatibility, and display capability. `isStreamingDevice` flags the pair the current stream is using; `isActive` flags the pair the shared auto-selector would bind a *new* stream to (the two differ when more than one pair is connected). Use it to tell two pairs of the same model apart.
* **Device pinning** — `startStreamSession(deviceId)` now streams from a *specific* pair: pass a `WearableDevice.id` from `getDevices()` to pin it (via the SDK's device selectors — `SpecificDeviceSelector` on Android, a filtered `AutoDeviceSelector` on iOS so disconnect detection keeps working), or `null` to auto-select. Throws a `PlatformException` with code `STREAM_ACTIVE` if a *different* device is already streaming — stop it first. The positional `deviceUUID` parameter on `startStreamSession`/`stopStreamSession`/`capturePhoto` is renamed to `deviceId` (positional, so existing call sites are unaffected).
* New types: `WearableDevice`, `WearableDeviceType`, `WearableLinkState`, `WearableCompatibility`. Non-breaking, drop-in addition.
* Example app: a "Paired devices" sheet (devices button, top-right) lists every paired pair and marks the streaming / selected one.
* Example app: a "Paired devices" sheet (devices button, top-right) lists every paired pair, marks the streaming pair, and lets you tap a pair (or "Automatic") to choose which one streams next.

## 0.5.3
* iOS: Fix first-registration device discovery and a `startStreamSession` hang after a failed start; surface genuine session errors.
Expand Down
23 changes: 13 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,18 +258,21 @@ The plugin follows Meta's integration lifecycle as documented in the [Meta Weara

### 3. Session (After registration and permissions)
- Once registered and permissions are granted, start a streaming session
- Call `MetaWearablesDat.startStreamSession(deviceUUID)` — returns a `textureId`
- Call `MetaWearablesDat.startStreamSession(deviceId)` — pass a `WearableDevice.id` from `getDevices()` to stream from a specific pair, or `null` to auto-select the active one. Returns a `textureId`
- Render the live video feed using Flutter's `Texture` widget with the returned ID
- Monitor session state via `MetaWearablesDat.streamSessionStateStream()`
- Monitor errors via `MetaWearablesDat.streamSessionErrorStream()`
- (Optional) Pin which pair streams by passing its `WearableDevice.id`; switching to a *different* pair while one is streaming throws a `PlatformException` with code `STREAM_ACTIVE` — stop the current stream first
- (Optional) Monitor device thermal level via `MetaWearablesDat.deviceStateStream()` to warn the user *before* a thermal error stops the stream
- (Optional) On `datAppOnTheGlassesUpdateRequired`, call `MetaWearablesDat.openDATGlassesAppUpdate()` to drive a "tap to update" UI
- Call `MetaWearablesDat.stopStreamSession(deviceUUID)` to end the session
- Call `MetaWearablesDat.stopStreamSession(deviceId)` to end the session

```dart
// Start streaming — returns a texture ID for zero-copy rendering
// Start streaming — returns a texture ID for zero-copy rendering.
// Pass null to auto-select the active pair, or a WearableDevice.id from
// getDevices() to stream from a specific pair.
final textureId = await MetaWearablesDat.startStreamSession(
deviceUUID,
null,
fps: 24,
streamQuality: StreamQuality.low,
videoCodec: VideoCodec.raw, // or VideoCodec.hvc1 (iOS only, supports background streaming)
Expand Down Expand Up @@ -313,12 +316,12 @@ for (final d in devices) {

// Capture a photo during streaming
final photo = await MetaWearablesDat.capturePhoto(
deviceUUID,
null,
format: PhotoCaptureFormat.jpeg, // or PhotoCaptureFormat.heic
);

// Stop streaming when done
await MetaWearablesDat.stopStreamSession(deviceUUID);
await MetaWearablesDat.stopStreamSession(null);
```

Video frames are pushed directly from native (CVPixelBuffer on iOS, SurfaceTexture on Android) to the Flutter engine — no JPEG encoding, no byte copying, no Dart-side decoding.
Expand Down Expand Up @@ -353,10 +356,10 @@ await MetaWearablesDat.enableBackgroundStreaming(
),
);

final textureId = await MetaWearablesDat.startStreamSession(deviceUUID);
final textureId = await MetaWearablesDat.startStreamSession(null);

// ...later, when you no longer need background execution:
await MetaWearablesDat.stopStreamSession(deviceUUID);
await MetaWearablesDat.stopStreamSession(null);
await MetaWearablesDat.disableBackgroundStreaming();
```

Expand Down Expand Up @@ -453,8 +456,8 @@ Meta gates registration on real glasses, so during development it's often handy
```yaml
# pubspec.yaml — add only in dev/staging builds
dependencies:
flutter_meta_wearables_dat: ^0.5.2
flutter_meta_wearables_dat_mock_device: ^0.5.2
flutter_meta_wearables_dat: ^0.6.0
flutter_meta_wearables_dat_mock_device: ^0.6.0
```

```dart
Expand Down
2 changes: 1 addition & 1 deletion agent/claude/rules/dat-flutter-conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
- Always use `MetaWearablesDat` static methods — never instantiate the class.
- Single import: `import 'package:flutter_meta_wearables_dat/flutter_meta_wearables_dat.dart';`
- All DAT operations are `async` — always `await` them.
- Pass `null` as `deviceUUID` to use AutoDeviceSelector (recommended for real devices). Only pass a specific UUID for mock devices.
- `startStreamSession(deviceId)`: `null` auto-selects the active pair; pass a `WearableDevice.id` from `getDevices()` to pin a specific pair (real or mock). Switching pairs while streaming throws `STREAM_ACTIVE` — stop first, then start.

## Lifecycle order

Expand Down
4 changes: 2 additions & 2 deletions agent/claude/skills/camera-streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ final errorSub = MetaWearablesDat.streamSessionErrorStream().listen((error) {

// Start streaming — returns texture ID
final textureId = await MetaWearablesDat.startStreamSession(
null, // null = AutoDeviceSelector (recommended for real devices)
null, // null = auto-select; or a WearableDevice.id from getDevices() to pin a pair
fps: 24,
streamQuality: StreamQuality.low,
videoCodec: VideoCodec.raw,
Expand Down Expand Up @@ -132,7 +132,7 @@ The stream switches its underlying subscription automatically when the active de

```dart
final photo = await MetaWearablesDat.capturePhoto(
null, // or specific deviceUUID
null, // or a WearableDevice.id from getDevices() to target a specific pair
format: PhotoCaptureFormat.jpeg, // or .heic (iOS only)
);
// photo.bytes — Uint8List of the image
Expand Down
4 changes: 2 additions & 2 deletions agent/claude/skills/mock-device-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ Since `flutter_meta_wearables_dat` 0.4.0 the mock APIs live in a separate option
```yaml
# pubspec.yaml
dependencies:
flutter_meta_wearables_dat: ^0.5.2
flutter_meta_wearables_dat_mock_device: ^0.5.2
flutter_meta_wearables_dat: ^0.6.0
flutter_meta_wearables_dat_mock_device: ^0.6.0
```

```dart
Expand Down
2 changes: 1 addition & 1 deletion agent/cursor/rules/dat-conventions.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ description: Coding conventions for flutter_meta_wearables_dat — API patterns,
## Video
- `startStreamSession()` returns `textureId` → render with `Texture(textureId: id)`.
- Never decode frames manually for display.
- Pass `null` as deviceUUID for real devices (AutoDeviceSelector), specific UUID only for mock devices.
- `startStreamSession(deviceId)`: pass `null` to auto-select the active pair, or a `WearableDevice.id` from `getDevices()` to pin a specific pair (real or mock). Switching to a different device while one is streaming throws `STREAM_ACTIVE` — stop first.

## Deep links
- Use `app_links` package. Forward every URI to `MetaWearablesDat.handleUrl(uri.toString())`.
Expand Down
4 changes: 2 additions & 2 deletions agent/cursor/rules/dat-mock-device.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ Mock APIs live in the optional add-on `flutter_meta_wearables_dat_mock_device` (
## Add the dev-only dependency
```yaml
dependencies:
flutter_meta_wearables_dat: ^0.5.0
flutter_meta_wearables_dat_mock_device: ^0.5.0
flutter_meta_wearables_dat: ^0.6.0
flutter_meta_wearables_dat_mock_device: ^0.6.0
```

```dart
Expand Down
6 changes: 5 additions & 1 deletion agent/cursor/rules/dat-streaming.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,18 @@ globs: "**/*stream*.dart,**/*video*.dart,**/*camera*.dart,**/*texture*.dart"
## Start session
```dart
final textureId = await MetaWearablesDat.startStreamSession(
null, // null = AutoDeviceSelector
null, // null = auto-select; or a WearableDevice.id (getDevices()) to pin a pair
fps: 24,
streamQuality: StreamQuality.low,
videoCodec: VideoCodec.raw,
);
// Render: Texture(textureId: textureId)
```

## Pinning a specific pair
- Pass a `WearableDevice.id` from `getDevices()` as the first arg to stream from that exact pair (implemented via `SpecificDeviceSelector` on Android and a filtered `AutoDeviceSelector` on iOS); `null` auto-selects.
- Calling `startStreamSession` for a *different* device while one is already streaming throws a `PlatformException` `STREAM_ACTIVE` — stop the current stream first.

## Subscribe to state/errors BEFORE starting
- `streamSessionStateStream()` — stopping(0), stopped(1), waitingForDevice(2), starting(3), streaming(4), paused(5)
- `streamSessionErrorStream()` — error codes: `thermalCritical`, `thermalEmergency`, `peakPowerShutdown`, `batteryCritical`, `hingesClosed`, `permissionDenied`, `datAppOnTheGlassesUpdateRequired` (call `openDATGlassesAppUpdate()` to recover), `dwaUnavailable`, plus device-session variants (`deviceThermalCritical` etc.).
Expand Down
9 changes: 5 additions & 4 deletions agent/github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,20 +12,21 @@ All methods are static on `MetaWearablesDat`. Single import: `import 'package:fl
2. `startRegistration()` → user confirms in Meta AI → `handleUrl(url)` deep link callback
3. `restartActiveDeviceMonitoring()` — call after registration (critical on Android)
4. `requestCameraPermission()` — Meta AI permission bottom sheet
5. `startStreamSession(deviceUUID, fps:, streamQuality:, videoCodec:)` → returns `textureId`
5. `startStreamSession(deviceId, fps:, streamQuality:, videoCodec:)` → returns `textureId` — pass a `WearableDevice.id` from `getDevices()` to pin a specific pair, or `null` to auto-select
6. Render: `Texture(textureId: textureId)` — zero-copy GPU rendering

### Key methods

- `registrationStateStream()` — RegistrationState: unavailable(0), available(1), registering(2), registered(3)
- `activeDeviceStream()` — bool, device availability
- `getDevices()` → `List<WearableDevice>` (id, name, type, linkState, compatibility, supportsDisplay, isActive, isStreamingDevice). Pass an `id` to `startStreamSession` to pin that pair; requesting a *different* device while one is streaming throws `PlatformException` `STREAM_ACTIVE`. Android throws `NOT_INITIALIZED` if called before Bluetooth permission.
- `streamSessionStateStream()` — stopping(0), stopped(1), waitingForDevice(2), starting(3), streaming(4), paused(5)
- `streamSessionErrorStream()` — StreamSessionError with code/message. Codes: thermalCritical, thermalEmergency, peakPowerShutdown, batteryCritical, hingesClosed, permissionDenied, deviceNotConnected, datAppOnTheGlassesUpdateRequired (recover via `openDATGlassesAppUpdate()`), dwaUnavailable, plus device-session variants (deviceThermalCritical etc.).
- `deviceStateStream()` — `Stream<DeviceState>` of live `ThermalLevel` (unknown, none, light, moderate, severe, critical, emergency, shutdown). Use to warn the user *before* a thermal error stops the stream.
- `openDATGlassesAppUpdate()` — opens the Meta AI app to the DAT-app-update screen. Pair with the `datAppOnTheGlassesUpdateRequired` error code to drive a "tap to update" UI.
- `videoStreamSizeStream()` — VideoStreamSize (width/height/aspectRatio) emitted on stream start and resolution changes
- `stopStreamSession(deviceUUID)` — end session
- `capturePhoto(deviceUUID, format:)` — PhotoCaptureFormat.jpeg or .heic
- `stopStreamSession(deviceId)` — end session
- `capturePhoto(deviceId, format:)` — PhotoCaptureFormat.jpeg or .heic
- `captureStreamFrame(textureId, width:, height:, format:)` — Dart-side rasterization for ML/OCR. Returns `null` while backgrounded.
- `enableBackgroundStreaming(androidNotification: BackgroundNotification(...))` — opt-in; call BEFORE `startStreamSession()` to keep the session alive while backgrounded or screen-locked. iOS + Android. `BackgroundNotification` is required on Android.
- `disableBackgroundStreaming()` — tears down the iOS `AVAudioSession` / stops the Android foreground service. Idempotent.
Expand All @@ -35,7 +36,7 @@ All methods are static on `MetaWearablesDat`. Single import: `import 'package:fl

Mock support lives in the optional add-on `flutter_meta_wearables_dat_mock_device` (since 0.4.0). Production builds that omit it skip `MWDATMockDevice` linkage and don't need `NSCameraUsageDescription` / `CAMERA`.

- Add `flutter_meta_wearables_dat_mock_device: ^0.5.2` to dev/staging `pubspec.yaml`.
- Add `flutter_meta_wearables_dat_mock_device: ^0.6.0` to dev/staging `pubspec.yaml`.
- Import: `import 'package:flutter_meta_wearables_dat_mock_device/flutter_meta_wearables_dat_mock_device.dart';`
- Optional bypass for registration/permission flows: `MetaWearablesDatMockDevice.configure(initiallyRegistered: true, initialPermissionsGranted: true)`
- Lifecycle: `MetaWearablesDatMockDevice.pairRayBanMeta()` → UUID, then `.powerOn(uuid)` + `.don(uuid)`, optionally `.setCameraFacing(uuid, CameraFacing.back)`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,17 +56,20 @@ internal class ActiveDeviceStreamHandler(
* double-subscribing. Matches the official CameraAccess sample's one-shot `startMonitoring()`
* pattern.
*/
fun restartMonitoring() {
fun restartMonitoring(force: Boolean = false) {
val sink = eventSink
if (sink == null) {
Log.d(TAG, "ActiveDeviceStream restartMonitoring — no sink yet, skipping")
return
}
if (job?.isActive == true) {
// `force` rebinds to a freshly-swapped selector even while a job is live
// (e.g. after a device-pin change). startCollecting() cancels the old job
// and re-reads the selector provider.
if (!force && job?.isActive == true) {
Log.d(TAG, "ActiveDeviceStream restartMonitoring — already collecting, skipping")
return
}
Log.d(TAG, "ActiveDeviceStream restartMonitoring — starting collection")
Log.d(TAG, "ActiveDeviceStream restartMonitoring — starting collection (force=$force)")
startCollecting(sink)
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,12 @@ internal class DeviceStateStreamHandler(
* Restart the active-device subscription. Called after BT permissions are
* granted so we don't miss the first device emission.
*/
fun restartMonitoring() {
fun restartMonitoring(force: Boolean = false) {
val sink = eventSink ?: return
if (outerJob?.isActive == true) return
// `force` rebinds to a freshly-swapped selector even while the outer job
// is live (e.g. after a device-pin change); startCollecting() cancels and
// re-reads the selector provider.
if (!force && outerJob?.isActive == true) return
if (!isInitialized()) return
startCollecting(sink)
}
Expand Down
Loading
Loading