Text generation

HuggingFaceTB/SmolLM2-360M-Instruct

Model ID: HuggingFaceTB/SmolLM2-360M-Instruct
Type: Text generation

Use this model

Generate text in a web app

Install the browser SDK:

npm install @wfloat/wfloat-web

Load the model and generate text locally in the browser:

import { loadLlmModel } from "@wfloat/wfloat-web";

const llm = await loadLlmModel("HuggingFaceTB/SmolLM2-360M-Instruct", {
  onProgress(event) {
    console.log(event.status, event.progress ?? "");
  },
});

const result = await llm.generate(
  "Write one calm sentence about local inference.",
  {
    maxTokens: 64,
    temperature: 0.7,
    seed: 0,
  },
);

console.log(result.text);

Use the onToken generation option when the interface should display tokens as they are produced.

Generate text in React Native

Install the React Native SDK:

npm install @wfloat/react-native-wfloat

Run pod install from the app’s ios directory. Then load the model and generate text on the device:

import { loadLlmModel } from '@wfloat/react-native-wfloat';

const llm = await loadLlmModel('HuggingFaceTB/SmolLM2-360M-Instruct', {
  onProgress(event) {
    console.log(event.status, event.progress ?? '');
  },
});

const result = await llm.generate({
  prompt: 'Write one calm sentence about local inference.',
  maxTokens: 64,
  temperature: 0.7,
  seed: 0,
  onToken(event) {
    console.log(event.text);
  },
});

console.log(result.text);

Use llm.chat({ messages }) for role-based conversations.

Generate text in Python

Install the Python SDK:

pip install wfloat

Load the model and generate text locally:

import wfloat

llm = wfloat.load_llm_model("HuggingFaceTB/SmolLM2-360M-Instruct")

try:
    result = llm.generate(
        "Write one calm sentence about local inference.",
        max_tokens=64,
        temperature=0.7,
        seed=0,
    )
    print(result.text)
finally:
    llm.close()

Use llm.chat(messages) for role-based conversations and on_token to receive generated text incrementally.