| ▲ | HowardStark 2 hours ago | |
Curious if anyone has done anything meaningful with SAM2 and streaming. SAM3 has built-in streaming support which is very exciting. I’ve seen versions where people use an in-memory FS to write frames of stream with SAM2. Maybe that is good enough? | ||
| ▲ | tom-in-july 12 minutes ago | parent [-] | |
The native support for streaming in SAM3 is awesome. Especially since it should also remove some of the memory accumulation for long sequences. I used SAM2 for tracking tumors in real-time MRI images. With the default SAM2 and loading images from the da, we could only process videos with 10^2 - 10^3 frames before running out of memory. By developing/adapting a custom version (1) based on a modified implementation with real (almost) stateless streaming (2) we were able to increase that to 10^5 frames. While this was enough for our purposes, I spend way too much time debugging/investigating tiny differences between SAM2 versions. So it’s great that the canonical version now supports streaming as well. (Side note: I also know of people using SAM2 for real-time ultrasound imaging.) 1 https://github.com/LMUK-RADONC-PHYS-RES/mrgrt-target-localiz... | ||