sodcalculator

10 Tech How-To

Trace App Failure from Local Setup to Server

Guide 10-9644 · Updated · Step by step

This guide walks you through a systematic method for tracing an application failure that appears only after deployment, even though it works locally. You will learn how to compare environments, isolate the failing layer, and pinpoint the root cause using logs, configuration checks, and targeted tests.

What You’ll Need

  • Access to the local development environment and the server (SSH, remote desktop, or cloud console).
  • Application logs from both environments (or the ability to enable them).
  • Deployment scripts or release notes showing what changed.
  • Basic command-line knowledge (terminal, shell).
  • Optional: a log aggregation tool or a simple text editor for comparing logs.

Step-by-Step

  1. Reproduce the failure on the server. Trigger the same action that fails in production. Note the exact error message, status code, or user-visible symptom. If you cannot reproduce it, gather as much context as possible from users or monitoring alerts.
  2. Check the server logs first. Look at the application log, web server log (e.g., Nginx, Apache), and system log (e.g., journalctl on Linux). Search for the timestamp of the failure. Identify the first error or exception that appears.
  3. Compare configuration files. Diff the environment variables, configuration files (e.g., .env, config/*.yml), and any feature flags between local and server. Pay special attention to database connection strings, API endpoints, file paths, and debug mode settings.
  4. Verify dependency versions. Run pip freeze (Python), npm list (Node.js), composer show (PHP), or the equivalent for your stack on both local and server. Look for differences in major or minor versions that could cause behavioral changes.
  5. Check environment-specific settings. Confirm that the server is using the correct environment (e.g., NODE_ENV=production, APP_ENV=prod). Some frameworks disable error display in production, so you may need to enable logging to a file.
  6. Test the network path. From the server, try to reach any external services the app depends on (database, cache, third-party API). Use curl or telnet to check connectivity and response times. A firewall or DNS issue can cause failures that don’t appear locally.
  7. Inspect file permissions and paths. Ensure the application has read/write access to required directories (uploads, logs, cache). Check that absolute paths are correct and that the server’s working directory matches the project root.
  8. Enable verbose logging temporarily. If logs are too sparse, increase log level to DEBUG or TRACE on the server, reproduce the issue, then revert. This often reveals the exact line of code or query that fails.
  9. Use a debugger or step-through on the server. If possible, run the application in a staging environment with a debugger attached (e.g., Xdebug, PyCharm remote debug). This is more invasive but can pinpoint the exact variable state.
  10. Isolate the failing layer. If the error is in the database, test queries directly. If it’s a frontend issue, open browser dev tools and check network requests. If it’s an API, use a REST client to call endpoints directly.
  11. Review recent deployments. Check the release notes or git log. Did the last deployment change a dependency, a configuration value, or a code path that could be related? Roll back the last change temporarily to test.
  12. Create a minimal reproduction script. Write a small script that exercises the failing code path with the server’s configuration. Run it locally with server-like settings to see if it fails there too.
  13. Document and fix. Once you find the root cause, apply the fix, test it on the server, and document the issue in your team’s knowledge base.

Common Mistakes

  • Assuming it’s a code issue. Often the code is fine; the problem is a missing environment variable or a different OS path separator.
  • Ignoring the web server layer. Nginx or Apache can return 502 or 504 errors that mask the actual application error. Always check the web server log first.
  • Comparing only code, not data. The server may have different data (e.g., a missing column or a null value) that causes a crash.
  • Forgetting to clear caches. Opcode caches (OPcache, APC) or application caches (Redis, Memcached) can serve stale code or data. Clear them after deploying changes.
  • Not checking disk space. A full disk can cause writes to fail silently or the app to crash. Check df -h on the server.

Tips

  • Use structured logging. Include request IDs, user IDs, and timestamps in logs to correlate events across services.
  • Mirror the production environment locally. Use Docker or a VM to replicate the server’s OS, PHP version, and extensions. This reduces surprises.
  • Automate configuration checks. Write a script that compares environment variables and dependency versions between local and server.
  • Monitor error tracking services. Tools like Sentry or Rollbar can capture exceptions with stack traces and context, even in production.
  • Keep a deployment checklist. Include steps like clearing caches, running migrations, and setting file permissions.

Troubleshooting

If you cannot find the error in the logs: Check if the application is logging to a different file than expected. Look at the framework’s logging configuration. Also, check if the log file is writable.

If the error only happens intermittently: Look for race conditions, timeouts, or resource exhaustion. Check memory usage, CPU, and database connection limits. Enable slow query logging.

If the server returns a generic 500 error: Temporarily set display_errors=1 in PHP or DEBUG=True in Django, but only on a staging server, not production. This will show the actual exception.

If the app works on the server but not locally: This is the reverse problem. Check if the server has a different PHP version, missing extensions, or stricter security settings (e.g., SELinux).

If you suspect a database issue: Run the exact queries the app makes, using the same credentials and database name. Check for collation or charset mismatches.

Frequently Asked Questions

Why does my app work locally but fail on the server?

Most often due to differences in environment variables, dependency versions, or file permissions. Use the steps above to compare systematically.

How do I enable detailed error reporting on a production server?

You should not expose errors to users, but you can enable logging to a file. For PHP, set display_errors=0 and log_errors=1. For Node.js, use a logging library like Winston.

What should I do if the logs show no errors at all?

Check if logging is correctly configured and that the log file is writable. Also check the web server logs—the error might be happening before the application is reached.

How can I compare configurations between local and server quickly?

Use a script to dump environment variables and configuration files, then run diff on them. Tools like Ansible or Docker Compose can also help standardize environments.

Is it safe to run a debugger on a production server?

Generally not recommended because it can expose sensitive data and slow down the server. Use a staging environment that mirrors production instead.

What if the failure is caused by a third-party service that behaves differently?

Test the API endpoint from the server using curl to see the exact response. Check if the server has a different IP allowlist or proxy settings.

Related

General steps for current mainstream versions; menus can differ by device and version. See the disclaimer or report an error.

More in this bed

The whole Tech How-To bed →