Password Manager Testing Criteria: How We Evaluate

Choosing a password manager is a security decision, and a good evaluation goes beyond reading a few reviews. This guide outlines a practical, repeatable testing methodology that covers security architecture, usability, feature completeness, and real-world reliability. You will learn what to check at each stage so you can confidently compare tools and pick one that fits your needs.

What You’ll Need

A list of candidate password managers (free trials or free tiers are fine).
Two or more devices (e.g., a laptop and a smartphone) to test cross-platform sync.
A test browser (Chrome, Firefox, Safari, or Edge) with a clean profile.
A test email account (not your primary) for account recovery checks.
A stopwatch or timer app for measuring response times.
A notes document or spreadsheet to record findings.

Step-by-Step

Define your threat model and requirements. Before testing, write down what you need: platforms (Windows, macOS, iOS, Android), browser support, features (password generator, autofill, breach monitoring, secure sharing), and any compliance requirements. This will be your scoring rubric.
Assess the security architecture. Read the official documentation and security whitepaper. Key questions: Is encryption zero-knowledge (client-side)? What encryption algorithm is used (AES-256 is standard)? Is the master password ever sent to the server? Is two-factor authentication (2FA) supported, and what methods (TOTP, hardware keys)? Check if the company has had any public security audits or published a bug bounty program.
Test the master password policy. Create an account with a strong master password (e.g., a passphrase). Try a weak one to see if it is rejected. Note the minimum length and complexity requirements. Then test the password recovery process: does it allow resetting the master password? If so, how does that affect access to stored data? (Zero-knowledge systems often require a recovery key or emergency sheet.)
Evaluate the password generator. Open the generator and note the options: length (up to 64 characters?), character sets (uppercase, lowercase, digits, symbols), and whether it excludes ambiguous characters. Generate several passwords and check that they are truly random (no patterns). Also test if the generator is available in the browser extension and mobile app.
Check autofill and capture behavior. Install the browser extension and log into a test website. Does it offer to save the password? Does autofill work on login forms, including those with multiple steps? Test on a few different sites (e.g., a standard form, a single-page app, a site with iframes). Note any sites where autofill fails or is slow.
Test cross-platform sync. Add a new entry on one device, then wait 30 seconds and check if it appears on another device. Repeat with a password change. Measure the sync delay. Also test offline access: disable internet on your phone and see if you can still view your vault (it should work if cached).
Examine the vault organization and search. Create several entries with different categories (login, credit card, secure note). Test the search function: does it find by URL, username, or notes? Try sorting and filtering. If the manager supports folders or tags, test moving entries between them.
Test the password sharing feature. If sharing is important to you, create a share link or invite a test account. Verify that the recipient can view the item without seeing the master password. Check if you can revoke access and if the share expires.
Review breach monitoring and security alerts. Use a test email address that you know has been in a breach (you can check with a public breach database). Add that email to the manager’s monitoring. See if it flags the breach and what actions it suggests (e.g., change password). Also test the password health report: it should flag reused, weak, or compromised passwords.
Test the mobile app. Install the app on your phone. Check biometric unlock (fingerprint/Face ID) and how quickly it locks. Test autofill on a mobile app (not just browser) – this often requires enabling accessibility services or the autofill API. Note any permissions the app requests.
Evaluate the import/export process. Export your vault (if you have one) from your current manager or a CSV file. Import it into the candidate. Check if fields map correctly (username, password, URL, notes). Then export from the candidate and verify the data is intact. This is critical for avoiding vendor lock-in.
Test customer support and documentation. Look for a help center, knowledge base, or live chat. Try to find an answer to a common question (e.g., “how to change master password”). Time how long it takes to get a response if you submit a ticket. Good documentation is a sign of a mature product.
Score and compare. Create a weighted scorecard with categories like Security (30%), Usability (25%), Features (20%), Reliability (15%), and Support (10%). Rate each candidate from 1-5 in each category, multiply by the weight, and sum. This gives you a quantitative comparison.

Common Mistakes

Only testing the free version – some features (like sharing or priority support) are paid-only, so you may miss critical limitations.
Not testing on all your devices – a manager that works great on desktop may have a clunky mobile app.
Ignoring the recovery process – if you lose your master password, you could lose all your data. Test the recovery flow before you commit.
Forgetting to check the company’s privacy policy – some managers collect telemetry or have a history of selling data. Read the policy carefully.
Assuming all password managers are the same – they differ in encryption, sync architecture, and feature sets. Always test your top candidates.

Tips

Use a dedicated test environment: a separate browser profile and a junk email account will keep your real data safe.
Test with realistic data – create entries that mimic your actual passwords (but don’t use real ones). This helps you see how the manager handles long URLs, special characters, and multiple accounts per site.
Time everything: how long does it take to save a new password, autofill, or sync? A few seconds’ difference adds up over a year.
Check the extension’s permissions – a good password manager should only request access to the sites you ask it to, not all sites.
Look for independent security audits – reputable managers publish them. If a manager has never had an audit, that’s a red flag.

Troubleshooting

Autofill not working on a site: Try manually filling the form and then saving the entry. Some sites use dynamic fields that the extension can’t detect. If it’s a critical site, check if the manager has a “fill on this site” override.
Sync delay: If changes don’t appear on another device, check that both devices are on the same account and that sync is enabled in settings. Sometimes you need to force a sync by pulling down to refresh or restarting the app.
Import errors: If a CSV import fails, try exporting from the old manager in a different format (e.g., CSV with different delimiters) or use the manager’s dedicated import tool if available.
Forgot master password: If the manager offers a recovery key, use it. If not, you may be locked out permanently – this is a design choice for security. Contact support to see if they have any options, but be prepared for a negative answer.
Browser extension crashes: Try disabling other extensions that might conflict. Also check if the extension is up to date. If it persists, reinstall the extension.

Frequently Asked Questions

What is the most important security feature to look for?

Zero-knowledge encryption is the most critical. This means the provider cannot see your passwords, even if their servers are breached. Look for AES-256 encryption and a clear statement that your master password is never stored or transmitted.

Should I use a cloud-based or local password manager?

Cloud-based managers are more convenient and offer sync across devices. Local-only managers (like KeePass) give you full control but require manual syncing. For most users, a reputable cloud-based manager with zero-knowledge encryption is a good balance of security and usability.

How often should I change my master password?

Only change it if you suspect it has been compromised or if you have shared it with someone. Otherwise, a strong, unique master password can be kept for years. Changing it frequently may lead to weaker choices.

Can a password manager be hacked?

No system is 100% unhackable, but a well-designed password manager with zero-knowledge encryption and strong 2FA makes it extremely difficult. The main risk is phishing – always ensure you are on the correct site before entering your master password.

What should I do if I lose my master password?

If the manager has a recovery key, use it. Otherwise, you will likely be locked out of your vault. Some managers offer an account recovery option that lets you reset the master password, but it may require a recovery code or a trusted device. Check the documentation before you need it.

Are free password managers safe?

Yes, many reputable password managers offer a free tier that includes core features like unlimited passwords and sync. They make money from premium plans for families or businesses. Just be sure to verify the security architecture is the same as the paid version.

General steps for current mainstream versions; menus can differ by device and version. See the disclaimer or report an error.

More in this bed

The whole Tech How-To bed →