linkding/README.md

# LinkDing

LinkDing is a minimal bookmarking application where you can paste links and get a list of links with title, description, and image. After a link is pasted, the page is scraped for metadata including the main image that is then displayed in the link list.  

## Features

- Paste links and get a list of links with title, description, and image
- Automatic metadata extraction
- Search functionality by title, description, and URL
- Organize links into custom lists
- Archive/unarchive links
- Public and private lists
- Modern, responsive web interface
- Support for JavaScript-heavy sites using Puppeteer
- Automatic fallback from HTTP scraping to browser rendering
- LDAP authentication support
- PostgreSQL database with automatic migrations

## Tech Stack

- **Backend**: Express.js (Node.js)
- **Frontend**: Vanilla JavaScript, HTML5, CSS3
- **Web Scraping**: Cheerio + Puppeteer (for JavaScript-heavy sites)
- **Database**: PostgreSQL with Sequelize ORM
- **Authentication**: LDAP (optional)

## Installation

### Prerequisites

- Node.js 18+ (or Docker)
- PostgreSQL 12+ (or Docker)
- Chromium/Chrome (for Puppeteer support, optional)

### Local Installation

1. Clone the repository or navigate to the project directory:
   ```bash
   cd linkding
   ```

2. Install dependencies:
   ```bash
   npm install
   ```

3. Set up environment variables:
   ```bash
   cp .env.example .env
   # Edit .env with your database configuration
   ```

4. Start PostgreSQL database and the application:
   ```bash
   make dev
   ```
   
   Or manually:
   ```bash
   # Start PostgreSQL (using docker-compose)
   docker compose -f docker-compose.dev.yaml up -d
   
   # Start the application
   npm start
   ```

5. Open your browser to `http://localhost:3000`

**Note**: On first startup, the application will:
- Create database tables automatically
- Migrate any existing JSON files (`data/links.json` and `data/lists.json`) to the database
- Rename migrated JSON files to `*.json.bak`

### Docker Installation

1. Set up environment variables:
   ```bash
   cp .env.example .env
   # Edit .env with your database configuration (or use defaults)
   ```

2. Use Docker Compose (recommended):
   ```bash
   docker-compose up -d
   ```
   
   This will start both PostgreSQL and the LinkDing application.

3. Access the application at `http://localhost:3000`

**Note**: The Docker Compose setup includes:
- PostgreSQL database with persistent volume
- LinkDing application container
- Automatic database initialization and migrations

## Usage

1. **Add a Link**: Paste a URL into the input field and click "Add Link"
2. **Search**: Use the search bar to filter links by title, description, or URL
3. **View Links**: Browse your saved links with images, titles, and descriptions
4. **Organize Links**: Create lists and assign links to them
5. **Archive Links**: Archive links to hide them from the main view
6. **Public Lists**: Make lists public to share them with unauthenticated users
7. **Delete Links**: Click the "Delete" button on any link card to remove it

## API Endpoints

### Links
- `GET /api/links` - Get all saved links (authenticated users see all, unauthenticated see only public lists)
- `GET /api/links/search?q=query` - Search links
- `POST /api/links` - Add a new link (body: `{ "url": "https://example.com" }`) - Requires authentication
- `PATCH /api/links/:id/archive` - Archive/unarchive a link (body: `{ "archived": true }`) - Requires authentication
- `PATCH /api/links/:id/lists` - Update link's lists (body: `{ "listIds": ["uuid1", "uuid2"] }`) - Requires authentication
- `DELETE /api/links/:id` - Delete a link by ID - Requires authentication

### Lists
- `GET /api/lists` - Get all lists (authenticated users see all, unauthenticated see only public)
- `POST /api/lists` - Create a new list (body: `{ "name": "List Name" }`) - Requires authentication
- `PUT /api/lists/:id` - Update a list (body: `{ "name": "New Name" }`) - Requires authentication
- `PATCH /api/lists/:id/public` - Toggle list public status (body: `{ "public": true }`) - Requires authentication
- `DELETE /api/lists/:id` - Delete a list by ID - Requires authentication

### Authentication
- `GET /api/auth/status` - Check authentication status
- `POST /api/auth/login` - Login with LDAP credentials (body: `{ "username": "user", "password": "pass" }`)
- `POST /api/auth/logout` - Logout

## Metadata Extraction

The application automatically extracts:

- **Title**: From Open Graph tags, JSON-LD structured data, or HTML `<h1>`/`<title>` tags
- **Description**: From meta tags, structured data, or page content
- **Images**: Prioritizes product container images, then meta tags, with smart fallbacks

### Image Extraction Priority

1. Product container images (`.product-container img`, etc.)
2. Product-specific image containers
3. Open Graph / Twitter Card meta tags
4. JSON-LD structured data
5. Generic product selectors
6. Fallback to meaningful images

## Environment Variables

See `.env.example` for a complete list of environment variables. Key variables include:

### Application
- `PORT` - Server port (default: 3000)
- `NODE_ENV` - Environment mode (production/development)

### Database
- `DATABASE_URL` - Full PostgreSQL connection string (e.g., `postgresql://user:password@host:port/database`)
- `DATABASE_SSL` - Enable SSL for database connection (true/false)
- `DB_HOST` - Database host (default: localhost)
- `DB_PORT` - Database port (default: 5432)
- `DB_NAME` - Database name (default: linkding)
- `DB_USER` - Database user (default: postgres)
- `DB_PASSWORD` - Database password (default: postgres)

### Session & Cookies
- `SESSION_SECRET` - Secret key for session encryption (change in production!)
- `SESSION_NAME` - Session cookie name (default: connect.sid)
- `COOKIE_SECURE` - Use secure cookies (default: true in production)
- `COOKIE_SAMESITE` - Cookie SameSite attribute (default: none for secure, lax otherwise)
- `COOKIE_DOMAIN` - Cookie domain (optional)
- `COOKIE_PATH` - Cookie path (default: /)
- `TRUST_PROXY` - Trust proxy headers (default: true)

### LDAP Authentication
- `LDAP_ADDRESS` - LDAP server address
- `LDAP_BASE_DN` - LDAP base distinguished name
- `LDAP_USER` - LDAP bind user
- `LDAP_PASSWORD` - LDAP bind password
- `LDAP_USERS_FILTER` - LDAP user search filter
- And more... (see `.env.example`)

### Puppeteer
- `CHROME_EXECUTABLE_PATH` - Path to Chrome/Chromium executable (for Puppeteer)

## Database

LinkDing uses PostgreSQL for data storage. The application automatically:

- **Creates tables** on first startup
- **Runs migrations** to keep the schema up to date
- **Migrates JSON files** if `data/links.json` or `data/lists.json` exist, then renames them to `*.json.bak`

### Migration System

The application includes a migration system for database schema changes:
- Migrations are stored in `migrations/` directory
- Migrations are automatically run on startup
- Each migration is tracked in the `SequelizeMeta` table

### Data Migration

If you have existing JSON files:
1. Place `links.json` and `lists.json` in the `data/` directory
2. Start the application
3. The files will be automatically migrated to PostgreSQL
4. Original files will be renamed to `links.json.bak` and `lists.json.bak`

## Troubleshooting

### Puppeteer Issues

If you encounter issues with Puppeteer:

1. **NixOS**: The app uses `puppeteer-core` and automatically detects system Chromium
2. **Docker**: Chromium is included in the Docker image
3. **Manual Setup**: Set `CHROME_EXECUTABLE_PATH` environment variable to your Chromium path

### 403 Errors

Some sites block automated requests. The app automatically:
- First tries HTTP requests with realistic headers
- Falls back to Puppeteer for JavaScript rendering if blocked
- Uses system Chromium for browser automation

## Development

### Using Make (Recommended)

```bash
# Start PostgreSQL and the application
make dev

# Start only PostgreSQL
make up

# Stop PostgreSQL
make down

# Stop and remove volumes (clean slate)
make clean
```

### Manual Development Setup

```bash
# Install dependencies
npm install

# Start PostgreSQL (using docker-compose)
docker compose -f docker-compose.dev.yaml up -d

# Run in development mode with auto-reload
npm run dev

# Start production server
npm start
```

### Database Management

The application uses Sequelize ORM with PostgreSQL. Database migrations are automatically run on startup. To manually manage the database:

- Connect to PostgreSQL: `psql -h localhost -U postgres -d linkding`
- Check migrations: Query the `SequelizeMeta` table
- View tables: `\dt` in psql

## License

ISC