LISAt: Language-Instructed Segmentation Assistant for Satellite Imagery

Jerome Quenum (University of California, Berkeley) · Wen-Han Hsieh (University of California, Berkeley) · Tsung-Han (Patrick) Wu (University of California, Berkeley) · Ritwik Gupta (University of California, Berkeley) · Trevor Darrell (Electrical Engineering & Computer Science Department) · David Chan (UC Berkeley)
annotationbenchmark modelcomplex user queriesdataset curationfoundation modelsgeospatial analysisgeospatial reasoningmultimodal pretrainingnatural-language promptsobject segmentationopen-domain reasoningreasoning segmentationremote-sensing imagerysegmentation modelsvision-language modelsvisual description tasks

Segmentation models can recognize a pre-defined set of objects in images. However, segmentation models capable of "reasoning" over complex user queries that implicitly refer to multiple objects of interest remain underexplored, especially in the geospatial domain. Recent advances in "reasoning segmentation"