What are flaky tests and how do you handle them?

Flaky tests are automated tests that produce inconsistent results - sometimes passing and sometimes failing without any code changes.

Common Causes of Flaky Tests

Timing Issues:

Insufficient waits for elements to load
Race conditions
Network latency

Test Dependencies:

Tests depending on execution order
Shared state between tests
Database state not cleaned up

External Dependencies:

Third-party API failures
Test environment instability
Resource contention

Concurrency Issues:

Parallel test execution conflicts
Shared resources

How to Fix Flaky Tests

1. Fix Timing Issues:

# Bad: Hard wait
time.sleep(5)

# Good: Explicit wait
wait = WebDriverWait(driver, 10)
element = wait.until(EC.element_to_be_clickable((By.ID, "button")))

2. Ensure Test Independence:

# Use setup and teardown
def setup_method(self):
    self.driver = webdriver.Chrome()
    # Clean state for each test

def teardown_method(self):
    self.driver.quit()
    # Clean up resources

3. Mock External Dependencies:

# Mock API responses instead of calling real APIs
@mock.patch('requests.get')
def test_api_call(mock_get):
    mock_get.return_value.json.return_value = {'status': 'success'}
    # Test with mocked response

4. Use Unique Test Data:

# Generate unique data for each test run
import uuid
test_email = f"test_{uuid.uuid4()}@example.com"

Best Practices

Identify and fix flaky tests immediately
Track flaky test metrics
Isolate tests from each other
Use proper waits (explicit over implicit)
Avoid hard-coded waits (Thread.sleep)
Clean up test data after each test
Run tests multiple times to identify flakiness
Quarantine consistently flaky tests