microsoft
/

Florence-2-large

Image-Text-to-Text

Model card Files Files and versions

Instructions to use microsoft/Florence-2-large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use microsoft/Florence-2-large with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="microsoft/Florence-2-large", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("microsoft/Florence-2-large", trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained("microsoft/Florence-2-large", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use microsoft/Florence-2-large with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "microsoft/Florence-2-large"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Florence-2-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/microsoft/Florence-2-large

How to use microsoft/Florence-2-large with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "microsoft/Florence-2-large" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Florence-2-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "microsoft/Florence-2-large" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Florence-2-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use microsoft/Florence-2-large with Docker Model Runner:
```
docker model run hf.co/microsoft/Florence-2-large
```

Resources

View closed (31)

How to get last hidden states?

#54 opened about 2 years ago by

embeddings

#51 opened about 2 years ago by

Batch: inefficient memory

#50 opened about 2 years ago by

Simple web server

#49 opened about 2 years ago by

What is the impact of reducing the size of the input image of the Davit image part

#48 opened about 2 years ago by

Update modeling_florence2.py

#47 opened about 2 years ago by

How to retrain the model with large max length?

#46 opened about 2 years ago by

prompt engineering in Florence model

#45 opened over 2 years ago by

vishalkatheriya

Assert config.vision_config.model_type == 'davit', 'only DaViT is supported for now'

#44 opened over 2 years ago by

LayerNorm missing in Statedict of DatViT L

#43 opened over 2 years ago by

Running Inference on Video

#41 opened over 2 years ago by

How to increase max token output ?

#40 opened over 2 years ago by

TypeError: Object of type Florence2LanguageConfig is not JSON serializable

#39 opened over 2 years ago by

Scores with OCR

#38 opened over 2 years ago by

Regarding Confidence Score

#36 opened over 2 years ago by

ValueError: Could not load model microsoft/Florence-2-large with any of the following classes.

#35 opened over 2 years ago by

hey, im new to huggingface,

#34 opened over 2 years ago by

Multimodal Tokenizer Question

#33 opened over 2 years ago by

Fine-tuning for multiple tasks strategy

#32 opened over 2 years ago by

Swift now supports inference, training, and deployment of the Florence models.

#31 opened over 2 years ago by

Question about continue pretrain on Florence large base model with more data

#30 opened over 2 years ago by

Update README.md

#29 opened over 2 years ago by

TypeError: Object of type Florence2LanguageConfig is not JSON serializable

#28 opened over 2 years ago by

unexpected keyword argument 'loss' - Fine tuning for CAPTIONS

#27 opened over 2 years ago by

Which image format is preferred? Error during inference

#26 opened over 2 years ago by

Not passing attention_mask in model.generate

#25 opened over 2 years ago by

Release Dataset

#24 opened over 2 years ago by

CPU inference

#22 opened over 2 years ago by

Please add to llama.cpp and ollama

#21 opened over 2 years ago by

Few shot prompting for object detection

#20 opened over 2 years ago by

How to Finetune?

#19 opened over 2 years ago by

Segment multiple same objects in the Image

#18 opened over 2 years ago by

load model from local files, disabling flash_attn, OSError: We couldn't connect to 'https://huggingface.co'

#15 opened over 2 years ago by

Safetensors weights?

#14 opened over 2 years ago by

Wrong bounding boxes

#13 opened over 2 years ago by

dynamic module utils error -- requires flash_attn

#11 opened over 2 years ago by

Segmentation Issue with identifying multiples, or contiguous objects

#10 opened over 2 years ago by

EnragedAntelope

What does the output coordinate format?

#9 opened over 2 years ago by

Running on macOS with Metal

#7 opened over 2 years ago by

How does this version compare with Paligemma

#6 opened over 2 years ago by