hexiaochun/img2video – Run with an API on Replicate

hexiaochun / img2video

输入图片和音频合并关键帧视频

Cold

Public
5.4K runs
CPU

Run with an API

Playground API Examples README Versions

Input

image

*file

Preview

输入图片

audio

*file

Preview

Video Player is loading.

Current Time 00:00:000

Duration 00:00:000

Loaded: 0%

Stream Type LIVE

Remaining Time 00:00:000

输入音频

video_type

string

关键帧运动方向

Default: "left_right"

Run this model in Node.js with one line of code:

npx create-replicate --model=hexiaochun/img2video

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run hexiaochun/img2video using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "hexiaochun/img2video:fbd74bb14b7b5d7a54d43132bff1e044d70b8e771e624b9839ac88d2ef7c6474",
  {
    input: {
      audio: "https://lf-bot-studio-plugin-resource.coze.cn/obj/bot-studio-platform-plugin-tos/sami/tts/3b1224641f2844a2baee0ea90e0f4311.mp3",
      image: "https://replicate.delivery/pbxt/Lb9isXusB16LpfAF3YAMtYP7sTKQNZyrsIaSDZUeuHBF9WPT/image.png",
      video_type: "left_right"
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run hexiaochun/img2video using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "hexiaochun/img2video:fbd74bb14b7b5d7a54d43132bff1e044d70b8e771e624b9839ac88d2ef7c6474",
    input={
        "audio": "https://lf-bot-studio-plugin-resource.coze.cn/obj/bot-studio-platform-plugin-tos/sami/tts/3b1224641f2844a2baee0ea90e0f4311.mp3",
        "image": "https://replicate.delivery/pbxt/Lb9isXusB16LpfAF3YAMtYP7sTKQNZyrsIaSDZUeuHBF9WPT/image.png",
        "video_type": "left_right"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run hexiaochun/img2video using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "hexiaochun/img2video:fbd74bb14b7b5d7a54d43132bff1e044d70b8e771e624b9839ac88d2ef7c6474",
    "input": {
      "audio": "https://lf-bot-studio-plugin-resource.coze.cn/obj/bot-studio-platform-plugin-tos/sami/tts/3b1224641f2844a2baee0ea90e0f4311.mp3",
      "image": "https://replicate.delivery/pbxt/Lb9isXusB16LpfAF3YAMtYP7sTKQNZyrsIaSDZUeuHBF9WPT/image.png",
      "video_type": "left_right"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-09-10T10:34:54.180152Z",
  "created_at": "2024-09-10T10:34:35.878000Z",
  "data_removed": false,
  "error": null,
  "id": "scz7yfyxwsrge0chvdztrwa60w",
  "input": {
    "audio": "https://lf-bot-studio-plugin-resource.coze.cn/obj/bot-studio-platform-plugin-tos/sami/tts/3b1224641f2844a2baee0ea90e0f4311.mp3",
    "image": "https://replicate.delivery/pbxt/Lb9isXusB16LpfAF3YAMtYP7sTKQNZyrsIaSDZUeuHBF9WPT/image.png",
    "video_type": "left_right"
  },
  "logs": "Moviepy - Building video 8053bd11-1b9d-4aa0-a4a0-8e4e736f9b0c.mp4.\nMoviePy - Writing audio in 8053bd11-1b9d-4aa0-a4a0-8e4e736f9b0cTEMP_MPY_wvf_snd.mp4\nchunk:   0%|          | 0/346 [00:00<?, ?it/s, now=None]\nchunk:  32%|███▏      | 110/346 [00:00<00:00, 1098.96it/s, now=None]\nchunk:  73%|███████▎  | 252/346 [00:00<00:00, 1286.57it/s, now=None]\nMoviePy - Done.\nMoviepy - Writing video 8053bd11-1b9d-4aa0-a4a0-8e4e736f9b0c.mp4\nt:   0%|          | 0/377 [00:00<?, ?it/s, now=None]\nt:   2%|▏         | 7/377 [00:00<00:05, 69.33it/s, now=None]\nt:   4%|▍         | 15/377 [00:00<00:05, 69.66it/s, now=None]\nt:   6%|▌         | 23/377 [00:00<00:04, 72.85it/s, now=None]\nt:   8%|▊         | 31/377 [00:00<00:04, 74.66it/s, now=None]\nt:  10%|█         | 39/377 [00:00<00:04, 75.08it/s, now=None]\nt:  12%|█▏        | 47/377 [00:00<00:04, 76.19it/s, now=None]\nt:  15%|█▍        | 55/377 [00:00<00:05, 60.76it/s, now=None]\nt:  16%|█▋        | 62/377 [00:00<00:05, 59.51it/s, now=None]\nt:  18%|█▊        | 69/377 [00:01<00:04, 61.74it/s, now=None]\nt:  20%|██        | 76/377 [00:01<00:05, 58.24it/s, now=None]\nt:  22%|██▏       | 83/377 [00:01<00:04, 58.96it/s, now=None]\nt:  24%|██▍       | 90/377 [00:01<00:04, 57.88it/s, now=None]\nt:  26%|██▌       | 98/377 [00:01<00:04, 61.50it/s, now=None]\nt:  28%|██▊       | 105/377 [00:01<00:04, 61.22it/s, now=None]\nt:  30%|██▉       | 112/377 [00:01<00:04, 61.79it/s, now=None]\nt:  32%|███▏      | 119/377 [00:01<00:04, 62.91it/s, now=None]\nt:  33%|███▎      | 126/377 [00:01<00:03, 64.67it/s, now=None]\nt:  35%|███▌      | 133/377 [00:02<00:03, 64.27it/s, now=None]\nt:  37%|███▋      | 140/377 [00:02<00:03, 63.17it/s, now=None]\nt:  39%|███▉      | 147/377 [00:02<00:03, 64.36it/s, now=None]\nt:  41%|████      | 155/377 [00:02<00:03, 65.72it/s, now=None]\nt:  43%|████▎     | 162/377 [00:02<00:03, 64.64it/s, now=None]\nt:  45%|████▍     | 169/377 [00:02<00:03, 65.05it/s, now=None]\nt:  47%|████▋     | 176/377 [00:02<00:03, 65.00it/s, now=None]\nt:  49%|████▊     | 183/377 [00:02<00:02, 65.77it/s, now=None]\nt:  50%|█████     | 190/377 [00:02<00:02, 65.93it/s, now=None]\nt:  52%|█████▏    | 197/377 [00:03<00:02, 65.41it/s, now=None]\nt:  54%|█████▍    | 204/377 [00:03<00:02, 65.84it/s, now=None]\nt:  56%|█████▌    | 211/377 [00:03<00:02, 64.45it/s, now=None]\nt:  58%|█████▊    | 218/377 [00:03<00:02, 65.22it/s, now=None]\nt:  60%|█████▉    | 226/377 [00:03<00:02, 67.65it/s, now=None]\nt:  62%|██████▏   | 233/377 [00:03<00:02, 67.59it/s, now=None]\nt:  64%|██████▎   | 240/377 [00:03<00:02, 66.60it/s, now=None]\nt:  66%|██████▌   | 247/377 [00:03<00:01, 65.63it/s, now=None]\nt:  67%|██████▋   | 254/377 [00:03<00:01, 65.66it/s, now=None]\nt:  69%|██████▉   | 261/377 [00:04<00:01, 65.21it/s, now=None]\nt:  71%|███████   | 268/377 [00:04<00:01, 62.98it/s, now=None]\nt:  73%|███████▎  | 276/377 [00:04<00:01, 65.32it/s, now=None]\nt:  75%|███████▌  | 284/377 [00:04<00:01, 67.73it/s, now=None]\nt:  77%|███████▋  | 291/377 [00:04<00:01, 66.98it/s, now=None]\nt:  79%|███████▉  | 298/377 [00:04<00:01, 60.60it/s, now=None]\nt:  81%|████████  | 305/377 [00:04<00:01, 60.40it/s, now=None]\nt:  83%|████████▎ | 312/377 [00:04<00:01, 60.21it/s, now=None]\nt:  85%|████████▍ | 320/377 [00:04<00:00, 63.52it/s, now=None]\nt:  87%|████████▋ | 327/377 [00:05<00:00, 64.83it/s, now=None]\nt:  89%|████████▊ | 334/377 [00:05<00:00, 64.26it/s, now=None]\nt:  90%|█████████ | 341/377 [00:05<00:00, 64.56it/s, now=None]\nt:  92%|█████████▏| 348/377 [00:05<00:00, 64.28it/s, now=None]\nt:  94%|█████████▍| 355/377 [00:05<00:00, 65.52it/s, now=None]\nt:  96%|█████████▋| 363/377 [00:05<00:00, 66.04it/s, now=None]\nt:  98%|█████████▊| 371/377 [00:05<00:00, 67.54it/s, now=None]\nMoviepy - Done !\nMoviepy - video ready 8053bd11-1b9d-4aa0-a4a0-8e4e736f9b0c.mp4",
  "metrics": {
    "predict_time": 8.927330179,
    "total_time": 18.302152
  },
  "output": "https://replicate.delivery/czjl/9CJBmLTskhI2BBSg6fA840AFmbUFsrDNPnYQe6K4uUjOLdbTA/8053bd11-1b9d-4aa0-a4a0-8e4e736f9b0c.mp4",
  "started_at": "2024-09-10T10:34:45.252822Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/scz7yfyxwsrge0chvdztrwa60w",
    "cancel": "https://api.replicate.com/v1/predictions/scz7yfyxwsrge0chvdztrwa60w/cancel"
  },
  "version": "fbd74bb14b7b5d7a54d43132bff1e044d70b8e771e624b9839ac88d2ef7c6474"
}

Generated in

8.9 seconds

Tweak it Share Report

Moviepy - Building video 8053bd11-1b9d-4aa0-a4a0-8e4e736f9b0c.mp4.
MoviePy - Writing audio in 8053bd11-1b9d-4aa0-a4a0-8e4e736f9b0cTEMP_MPY_wvf_snd.mp4
chunk:   0%|          | 0/346 [00:00<?, ?it/s, now=None]
chunk:  32%|███▏      | 110/346 [00:00<00:00, 1098.96it/s, now=None]
chunk:  73%|███████▎  | 252/346 [00:00<00:00, 1286.57it/s, now=None]
MoviePy - Done.
Moviepy - Writing video 8053bd11-1b9d-4aa0-a4a0-8e4e736f9b0c.mp4
t:   0%|          | 0/377 [00:00<?, ?it/s, now=None]
t:   2%|▏         | 7/377 [00:00<00:05, 69.33it/s, now=None]
t:   4%|▍         | 15/377 [00:00<00:05, 69.66it/s, now=None]
t:   6%|▌         | 23/377 [00:00<00:04, 72.85it/s, now=None]
t:   8%|▊         | 31/377 [00:00<00:04, 74.66it/s, now=None]
t:  10%|█         | 39/377 [00:00<00:04, 75.08it/s, now=None]
t:  12%|█▏        | 47/377 [00:00<00:04, 76.19it/s, now=None]
t:  15%|█▍        | 55/377 [00:00<00:05, 60.76it/s, now=None]
t:  16%|█▋        | 62/377 [00:00<00:05, 59.51it/s, now=None]
t:  18%|█▊        | 69/377 [00:01<00:04, 61.74it/s, now=None]
t:  20%|██        | 76/377 [00:01<00:05, 58.24it/s, now=None]
t:  22%|██▏       | 83/377 [00:01<00:04, 58.96it/s, now=None]
t:  24%|██▍       | 90/377 [00:01<00:04, 57.88it/s, now=None]
t:  26%|██▌       | 98/377 [00:01<00:04, 61.50it/s, now=None]
t:  28%|██▊       | 105/377 [00:01<00:04, 61.22it/s, now=None]
t:  30%|██▉       | 112/377 [00:01<00:04, 61.79it/s, now=None]
t:  32%|███▏      | 119/377 [00:01<00:04, 62.91it/s, now=None]
t:  33%|███▎      | 126/377 [00:01<00:03, 64.67it/s, now=None]
t:  35%|███▌      | 133/377 [00:02<00:03, 64.27it/s, now=None]
t:  37%|███▋      | 140/377 [00:02<00:03, 63.17it/s, now=None]
t:  39%|███▉      | 147/377 [00:02<00:03, 64.36it/s, now=None]
t:  41%|████      | 155/377 [00:02<00:03, 65.72it/s, now=None]
t:  43%|████▎     | 162/377 [00:02<00:03, 64.64it/s, now=None]
t:  45%|████▍     | 169/377 [00:02<00:03, 65.05it/s, now=None]
t:  47%|████▋     | 176/377 [00:02<00:03, 65.00it/s, now=None]
t:  49%|████▊     | 183/377 [00:02<00:02, 65.77it/s, now=None]
t:  50%|█████     | 190/377 [00:02<00:02, 65.93it/s, now=None]
t:  52%|█████▏    | 197/377 [00:03<00:02, 65.41it/s, now=None]
t:  54%|█████▍    | 204/377 [00:03<00:02, 65.84it/s, now=None]
t:  56%|█████▌    | 211/377 [00:03<00:02, 64.45it/s, now=None]
t:  58%|█████▊    | 218/377 [00:03<00:02, 65.22it/s, now=None]
t:  60%|█████▉    | 226/377 [00:03<00:02, 67.65it/s, now=None]
t:  62%|██████▏   | 233/377 [00:03<00:02, 67.59it/s, now=None]
t:  64%|██████▎   | 240/377 [00:03<00:02, 66.60it/s, now=None]
t:  66%|██████▌   | 247/377 [00:03<00:01, 65.63it/s, now=None]
t:  67%|██████▋   | 254/377 [00:03<00:01, 65.66it/s, now=None]
t:  69%|██████▉   | 261/377 [00:04<00:01, 65.21it/s, now=None]
t:  71%|███████   | 268/377 [00:04<00:01, 62.98it/s, now=None]
t:  73%|███████▎  | 276/377 [00:04<00:01, 65.32it/s, now=None]
t:  75%|███████▌  | 284/377 [00:04<00:01, 67.73it/s, now=None]
t:  77%|███████▋  | 291/377 [00:04<00:01, 66.98it/s, now=None]
t:  79%|███████▉  | 298/377 [00:04<00:01, 60.60it/s, now=None]
t:  81%|████████  | 305/377 [00:04<00:01, 60.40it/s, now=None]
t:  83%|████████▎ | 312/377 [00:04<00:01, 60.21it/s, now=None]
t:  85%|████████▍ | 320/377 [00:04<00:00, 63.52it/s, now=None]
t:  87%|████████▋ | 327/377 [00:05<00:00, 64.83it/s, now=None]
t:  89%|████████▊ | 334/377 [00:05<00:00, 64.26it/s, now=None]
t:  90%|█████████ | 341/377 [00:05<00:00, 64.56it/s, now=None]
t:  92%|█████████▏| 348/377 [00:05<00:00, 64.28it/s, now=None]
t:  94%|█████████▍| 355/377 [00:05<00:00, 65.52it/s, now=None]
t:  96%|█████████▋| 363/377 [00:05<00:00, 66.04it/s, now=None]
t:  98%|█████████▊| 371/377 [00:05<00:00, 67.54it/s, now=None]
Moviepy - Done !
Moviepy - video ready 8053bd11-1b9d-4aa0-a4a0-8e4e736f9b0c.mp4

Examples

View more examples

Run time and cost

This model costs approximately $0.0038 to run on Replicate, or 263 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on CPU hardware. Predictions typically complete within 38 seconds. The predict time for this model varies significantly based on the inputs.

Readme

This model doesn't have a readme.