Upload
experitest
View
117
Download
1
Tags:
Embed Size (px)
Citation preview
Cigniti & Experitest Co-webinar:
Mobile Banking Testing – Enabling Financial
Institutions to Succeed in the Digital Age
Jan 2015
Jan 2015
Guy Arieli
CTO
Experitest
Sachin Srivastava
BFSI COE Head
Cigniti Technologies
Speakers:
Agenda
Mobile banking industry: trends
Mobile banking industry: application type
Challenges in mobile banking testing
Experitest portfolio of tools for mobile banking
Cigniti framework for mobile banks
Case study
Demo
1
2
3
4
5
6
7
Trends in The Mobile Banking Industry
Banking – So Far
Branch
Banking
ATM
IVR
Internet Banking
Mobile Banking
Wearable
Gadgets
Changes leading the Banking Industry
Moore’s LawInternet of ThingsPredictive Banking
Image BankingOmni channel BankingAdoption of Social MediaCustomer Empowerment
Mobile Banking – The Evolution and Go Forward
7
115M Europeans will
bank using tablets in
2018, up from 19M in
2013
19% of US consumers
prefer to check their
bank balances via
mobile devices
70% of US consumers
are comfortable with
the increasing use of
virtual
communications in
banking in addition to
in-person
conversations
By 2017, more than
1B mobile subscribers
(15% of global
mobile subscribers)
will use mobile
banking
SMS Alerts
Account Queries
Funds
Transfer, Bill
Payment
Payment
Services
Loan request,
Service request
Service Convergence via Mobile Banking
ATM CALL CENTRE WEB CHANNEL
Mobile
• Cash withdrawal
• Account balance
• Transfers
• Deposit
• Bill payment
• Account balance
• Credit card balance
• Recent transactions
• Bill payment
• Lost credit card
• Account balance
• Bill payment
• Transfers
• Credit card balance
• Pay credit card
• Cash withdrawal
• Account balance
• Transfers
• Deposit
• Bill payment
• Credit card balance
• Recent transactions
• Pay credit card
Mobile converging most of the
services via technological
evolutions
Top Five Transactions via various Channels
Mobile Banking Functions
9
Account Information • Mini-statements and checking of account history, Balance Enquiry, Alerts on account activity or passing of pre- set thresholds and Monitoring of term deposits
• Access to Loan statements, Card Statements, Mutual funds/ equity statements and Insurance policy managementi
Bill Pay and Recharge
Services Utility bills, mobile bills, air ticket, bus ticket, DTH Recharge, Data Card Recharge, Movie Ticket etc.
and Bill Payment History
Transfer Funds
Via IMPS, Via NEFT and RTGS, Intra Bank transfers and VISA money transfers
Value added servicesCheque Book Request, Statement Request, Cheque Status Enquiry, IMPS Requests, Stop Cheque, Open a
Fixed Deposit, Open a Recurring Deposit , Apply Loans - Home, Car, Personal etc. and assisting customers
in location of ATM & Branches and various other Service Requests
Type of Application
Type of Application in the Banking industry
Customer Applications 1
Bank representative Applications 2
Web Applications3
Customer Applications
Focus on usability
Execution environment cannot be
controlled:
Different mobile OS
Different OS versions
Different device capabilities
Different network conditions.
1 Customer Applications
Bank Representative application
Bank Representative application
2
Usually a single device application
(usually iPad).
It's a rich in data
Requires longer training time
Web Applications
All the major bank have Web sites that
are adjust to mobile devices.
Web applications should be tested on
both Chrome and Safari with the
newest browsers versions.
Web Applications3
Challenges in Mobile Banking Apps Testing
Security
The Challenge
Strict security regulation on mobile application development and testing
Zero tolerant for security breaches
.
The Solution
Testing solution must be found inside the bank’s VPN, thus providing the same top-
security level as the bank’s other IT systems
Testing tool that does not require jail breaking or rooting of the devices under test
Long Tail of Customers
The Challenge Big banks service many users and need to support all OS , all OS version and devices
All the operating system in all versions need to be tested
include Blackberry and WP8-because also a small portion is a big number of users
.
The Solution
• Testing tool that support all OS: iOS, Android, Windows Phone and Blackberry
• Same tests should run – WITHOUT MODIFICATION – on all mobile OS
• Object repository that support native and web properties
• Hybrid app can be an amazing shortcut, in single development effort the bank can release an
application that will be supported on the majority of the OS, OS versions and devices.
• But as the user experience is extremely important, thy hybrid approach is only been used by the
small / mid banks.
Complex Data
.
The Solution
The Challenge the system under test configuration the user account configuration
How do I know that the information presented by the mobile application reflect
the bank backend databases?
Is the balance shown to the user is the actual balance?
Having a pool of accounts that can be used by the automation tests
Having a mechanism to lock a user for a specific test
Writing relative tests, the test will first learn the account status (like balance)
Closing the loop
.
The Solution
The Challenge How to close the automation loop?
To perform a test a user account details required
Usually creating new user is a very complex procedure that cannot be
performed as part of the automation flow
Terminal Emulation API that will enable the automation test to perform
query on the bank backend Mainframe.
Early Adoption
The Challenge Industry that constantly adopts new technologies.
Usage of new UI technologies and frameworks.
Support for new devices, new versions and new mobile OS
Short release cycles
.
The SolutionHighly spread agile development methodology
Continues integration for mobile
Mobile testing tool that can integrate to all major ALM assets
Continuous Integration For Mobile
Developers
Source Code
Repository
CI Server
YOURAPPExecuter Agent
Execute Test On Devices
x
Test Results
Need to scale
The Challenge Need to run regression and to execute the tests on 10's of devices
The number of tests and the execution time of single execution suite can take 10 Hrs.+
Number of engineers (automation and manual) can scale to hundreds.
.
The Solution Parallel/serial execution on numerous devices
Testing tool that enables:
High scalability
High Portability of test
Low maintenance of test
Top 10 banks span large continents
.
The Solution
On premise mobile device Lab that reside inside the organization VPN
The Challenge Testing team are spread globally
Ability to run tests on devices that are physically located in a different location
Due to security issues all devices and tests need to reside inside the bank’s VPN.
Fast speed response time when testing on a remote device.
Suitable Tools and Services
End-to-End Mobile Testing
App2Database
Legacy
App
Consumer Layer
Web services Web services
Middleware
App1
Apps, DB, Legacy
Layer
The Solution blueprint for end to end testing for Mobile Applications
• Integration testing of
Mobile Apps with
backend systems
(remove all stubbed
components)
• Validation of all the
Mobile UI and
Functionality with real
systems
2E2E Testing
• Web Services testing for project specific
functionality
• Web Services testing for all possible
positive and negative scenarios based on
the Interface Contract documents
• Web Services Testing at Unit level with
schema validation, data formats,
connectivity tests
1b Web Services Testing
• Mobile Application navigation testing with
wireframes documents and backend
stubbed
• Mobile Application UI testing with available
wireframes
1a Mobile Testing
Sample view of IT Architecture Proposed Test Model
Cigniti Enablers
Smart Tools-
Web Service
Validator
Automation
Centre of
Excellence
Mobile Solution
Centre on Cloud
Mobile Test
accelerators
(reusable test cases,
mind map)
Mobile Specific Tests
Non-Functional
Tests
Functional Tests
Mobile Specific Tests
Interruption Test Installation Test
Accessories Test Functional Testing
Upgrade Test
Drive Test
Memory Test
Performance Test
Network Type Test
Battery Performance Test
Carrier Testing
Geo-Fence Test
Details
Need for Mobile Test Automation
Challenges
• Ever shrinking timelines and minimal resources
• Testing of critical functionality on all the devices/OS is limited at the time of defect fixes or upgrades
• Defect leakage to production due to lack of sufficient
testing in Testing Phase
Attributes
• Manual Test suite development cost
• Automation Test suite development cost
• Cycle wise test execution cost
• Test suite maintenance cost
• Infrastructure cost
Influencing Factors
• Project schedules
• Processes
• Product / Application tech stack
• People expertise and skill set
• Price –investment on licenses etc.
Benefits
• Reduced Cost of Testing
• Reduced Time to Market
• Improved Product Quality
• Improved Test Coverage
• Improved Execution Cycles
End-to-End Product Suite - Manual, Automation, Monitoring, Load, Performer, Manager
• Manual testing tool
• Mini-recorder for automating short, repetitive, manual scenarios
• Report with screenshots of each step
• Manual to Automation Script Convertor
• Device Lab located onsite (in the customer VPN)
• Devices can be reserved and accessed by any remote user
• Used in conjunction with SeeTestAutomation/Manual for testing of remote devices
SeeTestManual
SeeTestCloud
• Automation tool
• Same test runs on different devices
• Plugs into all testing & CI environments:
SeeTestAutomation
SeeTest Network Virtualization
• Device performance under varying network conditions
• Simulating mobile network bandwidth latency and reception while on the move
SeeTestLoad• Real device generated mobile load
• Full integration to LoadRunner, Jmeter, etc.
Mobile Add-On for UFT• Full & complete integration into HP UFT:
• Record a test, edit it, run it, and view results
• Integration of object repository and more…
SeeTest ManagerFree Tool!
N
L
G
U
Demo
Case Study
• Client is a leading community bank in the US
• Key Requirement: Improve overall quality of mobile testing
and establish formal test processes
• Existing Application Environment: Applications built for iOS
and Android devices
• Engagement Model: Onsite – Offshore
Tools
• HP ALM
• SeeTest (from Experitest)
• Charles Proxy
• Cigniti Device Cloud
About the Client
Team Size and Duration
• Team Size: 8
• Duration: ongoing (expected 24 months +)
• Functionality Testing
• UI /Compatibility testing
• Usability testing
• Basic Application Performance Testing
• Drive Test
• Network Throttling tests
• Carrier Testing Type
Testing Types
Mobile Case Study - Leading Mobile Security ISV
Functional (UI/Compatibility) Testing Framework
Network
Real devices
Real User Perception via Test Automation Tool
Test Execution report
Graphs Reports Metrics
• Execution Reports and Dashboards for
making informed decisions
• Test application on carriers and Wi-Fi
networks
• Various types of UI, Compatibility,
Navigation, Usability Specific tests
• Customized Cigniti’s Test Automation
Framework
• Automated tests for any change on the build
• Best practices, entry and exit criteria-
improved test processes
Application Under Test
Challenges and Solution Provided
• Lack of visibility of product quality ahead of the release
into the production
• No template, guidelines, checklists, best practices to
ensure high levels of testing productivity and
effectiveness
• High testing costs and long test cycles with defects
slippage to production
• Long test cycles due to lack of test automation of
regression cycles
• Identified the application user scenarios and built a comprehensive
end to end mobile test strategy
• Selected Charles proxy to capture logs and perform Network
throttling tests
• Conducted tool feasibility analysis and selected Experitest SeeTest as
the automation tool for their native mobile application
• Leverage real devices from Cigniti’s Device Cloud to perform testing
• Built a framework with portability, re-usability, easy maintainability,
customized reporting engine and test data from various sources
Key Challenges Solutions Provided
Key Benefits Delivered
• Leveraged Cigniti mobile automation framework to optimize the total effort and reduced total effort by 40%
• Reduction in test cycle execution time by 60%
• Built scripts that could be leveraged across platforms (iOS and Android)
• 40% of carrier specific testing cost was reduced cost by leveraging Cigniti’s device cloud
Q&A
Thank you