Click here to load reader

Automate Legacy-System Testing: Easy, Reliable, and Extendible

  • View
    58

  • Download
    0

Embed Size (px)

Text of Automate Legacy-System Testing: Easy, Reliable, and Extendible

  1. 1. AUTOMATION TESTING LEGACY APPLICATIONS @EmanuilSlavov OF [email protected] emanuilslavov.com
  2. 2. THE UGLYTHE GOOD THE BAD
  3. 3. GREENFIELD vs. BROWNFIELD
  4. 4. GREENFIELD PROJECT
  5. 5. BROWNFIELD PROJECT BROWNFIELD PROJECT
  6. 6. WHY INVEST IN LEGACY SYSTEM?
  7. 7. TEAM HAPPINESS
  8. 8. LOW TEAM MORALE
  9. 9. Regression Bugs Fragile Software Slow Feedback Stupid Errors Repetitive Work Slow Progress
  10. 10. Quality software is team eort. It needs system thinking.
  11. 11. SHIFT LEFT
  12. 12. THE THREE PILLARS OF AUTOMATED TESTING
  13. 13. Static Code Analysis Unit Tests Black Box Tests
  14. 14. WHAT TO DO ABOUT IT
  15. 15. Start with basic acceptance tests
  16. 16. Functionality that makes money Must have functionality - compliance, security Repeating Manual Tests - Save Time Pareto Principle - 80/20
  17. 17. Your Tests Fast Reliable Maintainable
  18. 18. Do not test through the UI. (if possible)
  19. 19. result = RestClient.post( REGISTER_URL, user_details.to_json, {:content_type => :json} )
  20. 20. 800 test x 10 seconds = 2h 13min This saved us:
  21. 21. Set test data via API or DB.
  22. 22. Limit external dependencies calls. (talked about this last year)
  23. 23. Need to Call External System Automation Test? Talk to the real system No Fake the response Yes
  24. 24. Test should create the data they need.
  25. 25. Scenario: Client admin should not be able to view masters agencies Given master user And master creates agency And a client admin When client admin views master's agency Then client admin should get an error
  26. 26. Poll for results from API/DB operations.
  27. 27. sleeping(1).seconds.between_tries.failing_after(10).tries do result = some_operation raise 'No Data' if result['data'] == [] end
  28. 28. Run a test 20 times consecutively. Commit only if the test does not fail.
  29. 29. for i in {1..20}; do your_test; done
  30. 30. Automatically rerun failed tests.
  31. 31. Same static code checks for tests code as for production code.
  32. 32. CODE CHANGES
  33. 33. First Order of Business: Remove Unused Code
  34. 34. Remove the commented code Remove the code you know its not used Instrument the code to check whats really used
  35. 35. Second Order of Business: Stop The Rot
  36. 36. CONTINUOUS INTEGRATION
  37. 37. Run on every commit Max execution time: 5 min. Hook one by one all the checks Run longer tests periodically
  38. 38. Developers need to receive feedback about their new code within 5 minutes.
  39. 39. CHECKS ON COMMIT
  40. 40. The PHP Case
  41. 41. LINTER
  42. 42. php -l api/models/mobile_push_model.php PHP Parse error: api/models/mobile_push_model.php on line 61 Errors parsing api/models/mobile_push_model.php
  43. 43. HHVM
  44. 44. UnknownObjectMethod in file: api/models/mobile_push_model.php, line: 55, problem entry: $pusher->reallyUnsubscribeDevice ($params['user_id'], $params['device_id'], $actions)
  45. 45. STATIC CODE QUALITY
  46. 46. CYCLOMATIC COMPLEXITY function testPrint() { echo('Hello World'); } Complexity: 1 function testPrint($parameter) { if($parameter) { echo('Hello World'); } } Complexity: 2
  47. 47. Method complexity should be less than 10.
  48. 48. Complexity 82 Complexity 10 Constantly refactor to decrease complexity
  49. 49. Method size should be less than 100 lines (ideally less than 50).
  50. 50. Improve the code - then lower the threshold on commit check. Then repeat.
  51. 51. FIGHT LEGACY CODE WRITE UNIT TESTS
  52. 52. Written by Developers Fast, Independent Test Technical Aspects Cooperation between QA & Developers
  53. 53. [Demo]
  54. 54. 100% test coverage is not sucient!
  55. 55. Missing assertions Missing handling unlikely conditions Dont aim for specific coverage number
  56. 56. SECURITY TESTS
  57. 57. SQL Injection Detection (PHP and ADOdb) $dbConn->GetRow(SELECT * FROM users WHERE id = $user_id) $dbConn->GetRow(SELECT * FROM users WHERE id = ?, array($user_id))
  58. 58. Those errors can be caught with code analysis. No need to run slow whole application security scan.
  59. 59. There was no such tool. So we developed one.
  60. 60. github.com/emanuil/php-reaper
  61. 61. Scans PHP ADOdb code for SQL injections Command line Suitable for CI, on-commit tests Plans to expand, pull requests welcomed
  62. 62. MONITORING
  63. 63. Your second line of defence.
  64. 64. Show a lot with TV and Raspberry Pi.
  65. 65. Live Graphs + Deploys
  66. 66. Live Graphs + Deploys
  67. 67. CONCLUSION
  68. 68. utomat the most important functionalities Continuously improve static code quality Write unit tests for changed/new code Expand checks on commit Enable monitoring
  69. 69. RECOMMENDED READING

Search related