usefulweb.site Screenshot to Steps

Internet, AI & productivity

Screenshot to Steps

Turn a screenshot of an app or website into numbered steps for what you want to do.

PNG or JPG, max 5 MiB. Read with OCR on our server; not stored.

Try an example: Change password (labels pasted) · Find the settings

How it works

Upload a screenshot (or paste the visible labels) and say what you want to achieve. The text on screen is read with OCR, each label is classified as button, input field, navigation, link, checkbox or heading with its position, and the elements relevant to your goal are turned into numbered instructions.

  • OCR: tesseract (English) with word positions; words with confidence below 30 are dropped; words on one line are split into separate elements at gaps wider than 1.8× the text height.
  • Roles are assigned by rules: known action words → button; trailing colon or known field names → input field; short labels in the top 12% or known section names → navigation; URLs/"forgot…" → link; ☐/[ ] → checkbox; text 1.5× taller than median → heading.
  • Your goal is split into keywords and expanded with synonyms (e.g. password → security, login). Steps: open the best-matching navigation item (if no matching field is visible), fill visible input fields, tick relevant checkboxes, then click the primary button (Save/Continue/OK…).
  • Position is reported as a 3×3 grid region (top/middle/bottom × left/center/right).

Steps are derived only from text visible in the screenshot; icons without labels and hidden menus cannot be seen. Never upload screenshots that show passwords or card numbers.

For scripts and AI agents

  • JSON API: POST https://screenshottosteps.usefulweb.site/api/v1/calculate (same calculation as this page)
  • Interface description: OpenAPI 3.1 · AI guide: llms.txt
  • MCP server (Streamable HTTP): https://screenshottosteps.usefulweb.site/mcp · tool name screenshot_to_steps
Example request
curl -s -X POST https://screenshottosteps.usefulweb.site/api/v1/calculate \
  -H 'content-type: application/json' \
  -d '{"goal":"Change my password","uiText":"Account settings\nProfile\nSecurity\nNotifications\nCurrent password:\nNew password:\nConfirm password:\n[ ] Sign out of all other devices\nCancel\nSave"}'

Related tools

  • Error Message Translator – Paste a technical error and get the plain-English cause and troubleshooting steps.
  • Accessibility Fixer – Turn WCAG 2.2 problems in your HTML into concrete code fixes, grouped by effort.
Feedback