{
  "top_p": 0.9,
  "videos": [
    "https://replicate.delivery/pbxt/Lzl3gqYd6ExXDlkvvpAwtQWhWzIOtCiYW1ztjoHvaVVFNEzt/3325978-hd_1920_1080_24fps.mp4"
  ],
  "prompts": [
    "What is happening in this video?"
  ],
  "num_frames": 10,
  "temperature": 0.1,
  "max_new_tokens": 500
}

Install Replicate’s Node.js client library:

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run n1jl0091/video-llava-7b-hf_replicate_n1jl0091 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "n1jl0091/video-llava-7b-hf_replicate_n1jl0091:ff284eb7daa7ace568fe353efecc4728c1f1844771462d7ec3b4844741270ddf",
  {
    input: {
      top_p: 0.9,
      videos: ["https://replicate.delivery/pbxt/Lzl3gqYd6ExXDlkvvpAwtQWhWzIOtCiYW1ztjoHvaVVFNEzt/3325978-hd_1920_1080_24fps.mp4"],
      prompts: ["What is happening in this video?"],
      num_frames: 10,
      temperature: 0.1,
      max_new_tokens: 500
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Import the client:

import replicate

Run n1jl0091/video-llava-7b-hf_replicate_n1jl0091 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "n1jl0091/video-llava-7b-hf_replicate_n1jl0091:ff284eb7daa7ace568fe353efecc4728c1f1844771462d7ec3b4844741270ddf",
    input={
        "top_p": 0.9,
        "videos": ["https://replicate.delivery/pbxt/Lzl3gqYd6ExXDlkvvpAwtQWhWzIOtCiYW1ztjoHvaVVFNEzt/3325978-hd_1920_1080_24fps.mp4"],
        "prompts": ["What is happening in this video?"],
        "num_frames": 10,
        "temperature": 0.1,
        "max_new_tokens": 500
    }
)

print(output)

To learn more, take a look at the guide on getting started with Python.

Run n1jl0091/video-llava-7b-hf_replicate_n1jl0091 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "n1jl0091/video-llava-7b-hf_replicate_n1jl0091:ff284eb7daa7ace568fe353efecc4728c1f1844771462d7ec3b4844741270ddf",
    "input": {
      "top_p": 0.9,
      "videos": ["https://replicate.delivery/pbxt/Lzl3gqYd6ExXDlkvvpAwtQWhWzIOtCiYW1ztjoHvaVVFNEzt/3325978-hd_1920_1080_24fps.mp4"],
      "prompts": ["What is happening in this video?"],
      "num_frames": 10,
      "temperature": 0.1,
      "max_new_tokens": 500
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

[ "In this video, a woman is standing in a kitchen and preparing food.Ъ" ]

{
  "completed_at": "2024-11-18T18:03:53.648028Z",
  "created_at": "2024-11-18T18:01:42.148000Z",
  "data_removed": false,
  "error": null,
  "id": "007twqr00hrj20ck81r8068ffm",
  "input": {
    "top_p": 0.9,
    "videos": [
      "https://replicate.delivery/pbxt/Lzl3gqYd6ExXDlkvvpAwtQWhWzIOtCiYW1ztjoHvaVVFNEzt/3325978-hd_1920_1080_24fps.mp4"
    ],
    "prompts": [
      "What is happening in this video?"
    ],
    "num_frames": 10,
    "temperature": 0.1,
    "max_new_tokens": 500
  },
  "logs": "Starting prediction for /tmp/tmpt59elgyh3325978-hd_1920_1080_24fps.mp4 at 18:03:50\nUsing 10 frames\nExpanding inputs for image tokens in Video-LLaVa should be done in processing. Please add `patch_size` and `vision_feature_select_strategy` to the model's processing config or set directly with `processor.patch_size = {{patch_size}}` and processor.vision_feature_select_strategy = {{vision_feature_select_strategy}}`. Using processors without these attributes in the config is deprecated and will throw an error in v4.44.\nExpanding inputs for image tokens in Video-LLaVa should be done in processing. Please add `patch_size` and `vision_feature_select_strategy` to the model's processing config or set directly with `processor.patch_size = {{patch_size}}` and processor.vision_feature_select_strategy = {{vision_feature_select_strategy}}`. Using processors without these attributes in the config is deprecated and will throw an error in v4.47.\nTotal prediction time for /tmp/tmpt59elgyh3325978-hd_1920_1080_24fps.mp4: 3.42s",
  "metrics": {
    "predict_time": 4.072873206,
    "total_time": 131.500028
  },
  "output": [
    "In this video, a woman is standing in a kitchen and preparing food.Ъ"
  ],
  "started_at": "2024-11-18T18:03:49.575154Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/007twqr00hrj20ck81r8068ffm",
    "cancel": "https://api.replicate.com/v1/predictions/007twqr00hrj20ck81r8068ffm/cancel"
  },
  "version": "ff284eb7daa7ace568fe353efecc4728c1f1844771462d7ec3b4844741270ddf"
}

Generated in

4.1 seconds

Tweak it Iterate in playground Share Report

Want to make some of these yourself?

Run this model