Setup Guide

1Architecture Overview

Simple Stock Screener is a CodeIgniter 4 application that tracks the top 5 biggest stock gainers, biggest losers, and most active stocks by volume each trading day. It uses a cache-first architecture: market data is collected via scheduled cron jobs (not triggered by user visits) and stored in a MySQL database. All page loads and API requests are served from cached database records.

Data collection schedule:

• Market open snapshot — 9:45 AM Eastern Time (15 minutes after the 9:30 AM NYSE open)
• Market close snapshot — 4:15 PM Eastern Time (15 minutes after the 4:00 PM NYSE close)

Each collection job makes exactly 3 API requests to the Yahoo Finance API via RapidAPI:

1. Get movers — fetches top gainers, losers, and most active stocks
2. Get quotes — fetches accurate pricing for all symbols returned
3. Get spark data — fetches 5-day price history for mini chart rendering

With 2 collections per day (6 API calls) and a 500-request monthly limit, the system uses roughly 130 calls/month, leaving headroom for testing and manual runs.

2Server Requirements

3Server Setup

The production environment runs on RunCloud (a server management panel for DigitalOcean/Vultr/AWS).

Set the server timezone

The server must use the America/New_York timezone so that cron jobs fire at the correct market-relative times (EST in winter, EDT in summer). This is handled automatically by the OS timezone setting.

sudo timedatectl set-timezone America/New_York
timedatectl  # Verify

Install required PHP extensions

# Ubuntu/Debian (adjust php8.3 to your version)
sudo apt install php8.3-cli php8.3-mysql php8.3-curl php8.3-mbstring \
    php8.3-intl php8.3-xml php8.3-zip

# Verify
php -m | grep -iE "curl|json|mysqli|mbstring|intl|openssl"

Apache configuration

Enable mod_rewrite and point the virtual host's DocumentRoot to the public/ directory.

sudo a2enmod rewrite
sudo systemctl restart apache2

The included public/.htaccess handles URL rewriting, directory index blocking, and trailing-slash removal. Make sure your Apache config allows .htaccess overrides:

<Directory /path/to/project/public>
    AllowOverride All
    Require all granted
</Directory>

Nginx alternative

If using Nginx instead of Apache, use this server block:

server {
    listen 443 ssl;
    server_name yourdomain.com;
    root /path/to/project/public;
    index index.php;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        fastcgi_pass unix:/run/php/php8.3-fpm.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }

    location ~ /\. {
        deny all;
    }
}

4Database Setup

Create a MySQL/MariaDB database and user:

mysql -u root -p

CREATE DATABASE your_database_name CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
CREATE USER 'your_db_user'@'localhost' IDENTIFIED BY 'your_secure_password';
GRANT ALL PRIVILEGES ON your_database_name.* TO 'your_db_user'@'localhost';
FLUSH PRIVILEGES;
EXIT;

The database charset must be utf8mb4 with collation utf8mb4_general_ci to properly handle company names with special characters.

5Application Installation

Clone or upload the project

# Clone from repository
git clone <repository-url> /path/to/project

# Or upload via SFTP/SCP to the web server

Install PHP dependencies

cd /path/to/project
composer install --no-dev --optimize-autoloader

The --no-dev flag skips testing packages (PHPUnit, Faker) in production. The framework version is CodeIgniter 4.6.1.

6Environment Configuration (.env)

Copy the example env file and edit it. The .env file must be in the project root (same level as app/, public/, and composer.json).

cp env .env
nano .env

Set the following values:

#--------------------------------------------------------------------
# ENVIRONMENT
#--------------------------------------------------------------------
CI_ENVIRONMENT = production

#--------------------------------------------------------------------
# APP
#--------------------------------------------------------------------
app.baseURL = 'https://yourdomain.com/'

#--------------------------------------------------------------------
# DATABASE
#--------------------------------------------------------------------
database.default.hostname = localhost
database.default.database = your_database_name
database.default.username = your_db_user
database.default.password = your_secure_password
database.default.DBDriver = MySQLi
database.default.DBPrefix =
database.default.port = 3306

#--------------------------------------------------------------------
# STOCK MARKET API (RapidAPI - Yahoo Finance)
#--------------------------------------------------------------------
api.rapidapi.host = yh-finance.p.rapidapi.com
api.rapidapi.key = YOUR_RAPIDAPI_KEY_HERE
api.rapidapi.timeout = 30
api.rapidapi.max_retries = 3

# Rate limiting
api.rate_limit.requests_per_second = 5
api.rate_limit.requests_per_month = 500
api.rate_limit.daily_limit = 17

# Data collection schedule
api.collection.enabled = true
api.collection.market_open_time = 09:30
api.collection.market_close_time = 16:00
api.collection.timezone = America/New_York

Application timezone

The app timezone is set in app/Config/App.php line 136 and must be America/New_York. This is already configured — do not change it. It ensures all date() calls throughout the application use Eastern Time, matching NYSE market hours.

7RapidAPI Setup (Yahoo Finance)

The application uses the Yahoo Finance API on RapidAPI as its sole external data source.

Step 1: Create a RapidAPI account

Go to rapidapi.com and create an account.

Step 2: Subscribe to the Yahoo Finance API

Navigate to the YH Finance API page and subscribe to a plan. The Basic (free) plan provides 500 requests/month, which is sufficient for this application (uses ~130 requests/month with the default schedule).

Step 3: Get your API key

After subscribing, your API key will be displayed on the API page under Header Parameters > X-RapidAPI-Key. Copy this key into the .env file as api.rapidapi.key.

API endpoints used

8Database Migrations

Run the database migrations to create the required tables:

cd /path/to/project
php spark migrate

This creates two tables:

• market_snapshots — stores all stock data (prices, volume, spark charts)
• api_usage — tracks daily API request counts for rate limiting

See the Database Schema section below for full column details.

9File Permissions

Set proper ownership and permissions for the writable directory:

# Set the web server user as owner (adjust user/group for your setup)
sudo chown -R www-data:www-data /path/to/project/writable
sudo chmod -R 775 /path/to/project/writable

# Secure the .env file
chmod 600 /path/to/project/.env

The writable/ directory contains these subdirectories:

writable/
  cache/       # Framework cache files
  logs/        # Application logs (log-YYYY-MM-DD.log)
  session/     # PHP session storage
  debugbar/    # Debug toolbar data (development only)
  uploads/     # Reserved for future use

10Cron Jobs

The cron jobs are the heart of the application. Without them, no data is collected. These must be configured on the server.

Edit the crontab

crontab -e

Add these three entries

# Market open snapshot (9:45 AM ET, Monday-Friday)
# Runs 15 minutes after NYSE opens at 9:30 AM
45 9 * * 1-5 cd /path/to/project && php spark datacollection:collect >> /dev/null 2>&1

# Market close snapshot (4:15 PM ET, Monday-Friday)
# Runs 15 minutes after NYSE closes at 4:00 PM
15 16 * * 1-5 cd /path/to/project && php spark datacollection:collect >> /dev/null 2>&1

# Weekly cleanup (Sunday at 2:00 AM)
# Deletes market data older than 30 days
0 2 * * 0 cd /path/to/project && php spark datacollection:cleanup >> /dev/null 2>&1

Cron schedule breakdown

Daily API usage: 6 requests (3 per collection × 2 collections).
Monthly API usage: ~130 requests (22 trading days × 6), well within the 500/month limit.

11Testing the Installation

After completing all setup steps, verify the installation:

Step 1: Test the API connection

php spark datacollection:test

Expected output:

Testing data collection system...
Timestamp: 2026-04-14 15:30:00

Testing API connection...
✓ API connection successful

Testing data collection process...
✓ Data collection test successful
- API Response: Yes
- Data Structure Valid: Yes

Step 2: Run a manual data collection

php spark datacollection:collect

This will make 3 API calls and populate the database. You should see output listing gainers, losers, active stocks collected, quotes updated, and spark charts updated.

Step 3: Check the statistics

php spark datacollection:stats

Verify that records were collected and API usage was tracked.

Step 4: Load the website

Open your domain in a browser. The homepage should display the three data sections (Biggest Gainers, Biggest Losers, Most Active) with the data you just collected.

Step 5: Test the health endpoint

curl https://yourdomain.com/api/health

Should return a JSON response with "status": "ok" and a database connection confirmation.

12CLI Commands Reference

All commands are run from the project root directory.

13API Endpoints

The application exposes a REST API for programmatic access to the cached data.

14API Rate Limits

The Yahoo Finance API on RapidAPI enforces the following limits on the Basic (free) plan. The application tracks and enforces these internally.

API usage is tracked in the api_usage database table with one row per day. The system will refuse to make API calls if daily or monthly limits are reached.

15Database Schema

Table: market_snapshots

Indexes: Primary key on id, composite key on (snapshot_time, data_type), key on symbol.

Table: api_usage

16Directory Structure

project-root/
├── app/
│   ├── Config/
│   │   ├── App.php              # App timezone, base URL, locale
│   │   ├── Database.php         # Database connection settings
│   │   ├── Filters.php          # HTTP filters (CSRF, etc.)
│   │   ├── Routes.php           # All URL routes
│   │   └── Paths.php            # Directory path configuration
│   ├── Controllers/
│   │   ├── BaseController.php   # Abstract base controller
│   │   ├── Home.php             # Homepage (market overview)
│   │   ├── Historical.php       # Historical data page
│   │   ├── Pages.php            # Static pages (about, docs)
│   │   ├── Api.php              # REST API endpoints
│   │   └── DataCollectionCommand.php  # CLI commands for data collection
│   ├── Libraries/
│   │   ├── ApiService.php       # RapidAPI HTTP client and rate limiting
│   │   └── DataCollectionService.php  # Data collection orchestration
│   ├── Models/
│   │   ├── MarketDataModel.php  # market_snapshots table queries
│   │   └── ApiUsageModel.php    # api_usage table queries
│   ├── Database/
│   │   └── Migrations/          # Database migration files
│   └── Views/
│       ├── layout/
│       │   └── main.php         # Base HTML layout (nav, footer, scripts)
│       ├── home/
│       │   ├── index.php        # Market data view (shared: home + historical)
│       │   └── partials/
│       │       └── stock_table.php  # Stock data table partial
│       └── pages/
│           ├── about.php        # About page
│           └── docs.php         # This setup guide
├── public/
│   ├── index.php                # Front controller (entry point)
│   ├── .htaccess                # Apache rewrite rules
│   ├── favicon.ico              # Browser favicon (ICO)
│   ├── favicon.svg              # Browser favicon (SVG)
│   └── robots.txt               # Search engine crawler rules
├── writable/
│   ├── cache/                   # Framework cache
│   ├── logs/                    # Application logs
│   ├── session/                 # Session storage
│   └── uploads/                 # File uploads (reserved)
├── vendor/                      # Composer dependencies (auto-generated)
├── .env                         # Environment config (DO NOT commit)
├── composer.json                # PHP dependency definitions
├── spark                        # CodeIgniter CLI entry point
└── cron-setup.txt               # Cron job reference

17Maintenance

Monitor API usage

php spark datacollection:stats

Run this periodically to verify cron jobs are executing and API limits are not being exceeded.

Check application logs

# View the most recent log
ls -t writable/logs/ | head -1 | xargs -I {} cat writable/logs/{}

# Tail logs in real time
tail -f writable/logs/log-$(date +%Y-%m-%d).log

Manual data collection

If a cron job was missed (e.g., server restart), you can run a manual collection:

php spark datacollection:collect

Database backups

# Full database backup
mysqldump -u your_db_user -p your_database_name > backup_$(date +%Y%m%d).sql

# Market data only
mysqldump -u your_db_user -p your_database_name market_snapshots api_usage > market_backup_$(date +%Y%m%d).sql

Update dependencies

composer update --no-dev
php spark migrate  # Run any new migrations after updating

Data retention

The weekly cleanup cron deletes records older than 30 days. To change the retention period, modify the cleanup cron command to include --days=N (e.g., --days=90 for 90-day retention).

18Troubleshooting

No data on the homepage

• Run php spark datacollection:collect manually to populate the database.
• Check that the cron jobs are configured: crontab -l
• Verify the API key is correct in .env

API connection failures

• Run php spark datacollection:test to diagnose.
• Verify api.rapidapi.key in .env is valid and the subscription is active.
• Check that the PHP curl extension is installed: php -m | grep curl
• Check that outbound HTTPS is not blocked by a firewall.

Rate limit exceeded

• Run php spark datacollection:stats to check current usage.
• The monthly limit resets on the 1st of each month (per RapidAPI).
• Reduce collection frequency if needed (e.g., once daily instead of twice).

Database connection errors

• Verify credentials in .env match the actual database user/password.
• Ensure the database exists: mysql -u your_user -p -e "SHOW DATABASES;"
• Check that MySQL/MariaDB is running: systemctl status mysql

Cron jobs not running

• Verify the crontab: crontab -l
• Check the server timezone: timedatectl (must be America/New_York)
• Test the command manually from the project root.
• Check logs for errors after the scheduled time.

Blank page or 500 error

• Check writable/logs/ for the latest error log.
• Ensure writable/ directory is writable by the web server user.
• Verify CI_ENVIRONMENT = production is set in .env
• Run composer install to ensure all dependencies are present.