Chapter 10: Binary Data - When the AI Sends Images

A modern LLM can not only read, it can also "see". MCP allows servers to transfer not just text but binary data such as images directly into the context of the model. In this chapter we learn how that works technically.

The ImageContent Object

When a tool or a resource wants to return an image, it uses the ImageContent object. Unlike text, here the image is transferred Base64-encoded.

Example of a response with an image:

{
  "content": [
    {
      "type": "image",
      "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJ...",
      "mimeType": "image/png"
    }
  ]
}

Why Base64?

Because MCP is based on JSON-RPC and JSON is a text format, we cannot send "raw" binary data. Base64 converts the bits and bytes of the image into a string that can be safely transported inside a JSON string.

Implementation in Go

The Go SDK takes part of the encoding off our hands. You only have to supply the byte array of your image:

func handleGetImage(ctx context.Context, request *mcp.CallToolRequest, args any) (*mcp.CallToolResult, any, error) {
    // Load or generate the image
    imgBytes, _ := os.ReadFile("chart.png")

    return &mcp.CallToolResult{
        Content: []mcp.Content{
            &mcp.ImageContent{
                Data:     imgBytes,
                MIMEType: "image/png",
            },
        },
    }, nil, nil
}

Testing binary data with `mcp-tester`

Testing binary data is a challenge, because you cannot simply "see" images in the terminal. Our mcp-tester solves this in two ways:

Direct call (call): The tester recognises the binary data and prints a summary: Content 0 (Image): image/png data, size 12345 bytes.
Script mode (test): When a script calls a tool that returns an image, the mcp-tester automatically saves the image as a file in the current directory (e.g. get_image_0.png).

This way you can run the test and then simply open the file to check the result.

Outlook: Other Media Types

Although images (PNG, JPEG) are used most often, the protocol theoretically also allows other types such as audio. The logic stays the same: the data is transferred as a Base64 blob in the content array and presented by the client to the multimodal model.

← Chapter 9: Return Values | Table of Contents | Next Chapter: Real-Time Feedback →