Module reader

Main orchestration module for web page to markdown conversion.

This is the primary entry point for the mdfetch library. It combines all the individual components (HTTP fetching, content extraction, markdown conversion) into a single, easy-to-use pipeline.

The main function readURL handles the complete workflow:

Fetches HTML from the provided URL (with retries and timeout)
Converts all relative URLs to absolute URLs
Extracts readable content using Mozilla Readability
Converts the content to markdown using Turndown with GitHub Flavored Markdown
Returns content in all three formats (HTML, text, markdown) with metadata

Example

import { readURL } from 'mdfetch';

// Basic usage
const result = await readURL('https://example.com/article');
console.log(result.title);        // "Article Title"
console.log(result.markdown);     // "# Article Title\n\nContent..."
console.log(result.plainText);    // "Article Title Content..."
console.log(result.byline);       // "John Doe"

// With custom options
const result = await readURL('https://slow-site.com/article', {
  timeout: 60000,
  retries: 5,
  retryDelay: 2000
});

Classes

ReaderError

Functions

readURL

Module reader

Example

Classes

Functions

Settings

On This Page