fix(driver): fixed issue where the connection attempt of Flutter driver to a running app would not retry before throwing a connection error

feat(config): added a before `onBeforeFlutterDriverConnect` and after `onAfterFlutterDriverConnect` driver connection method property to the test configuration `FlutterTestConfiguration` to enable custom logic before and after a driver connection attempt.
This commit is contained in:
Jon Samwell 2020-05-11 12:20:35 +10:00
parent 2205ce1215
commit aad6af0ab1
7 changed files with 216 additions and 156 deletions

View File

@ -1,3 +1,7 @@
## [1.1.8+2] - 11/05/2020
* Fixed issue where the connection attempt of Flutter driver would not retry before throwing a connection error. This was causing an error on some machines trying to connect to an Android emulator (x86 & x86_64) that runs the googleapis (see https://github.com/flutter/flutter/issues/42433)
* Added a before `onBeforeFlutterDriverConnect` and after `onAfterFlutterDriverConnect` Flutter driver connection method property to the test configuration `FlutterTestConfiguration` to enable custom logic before and after a driver connection attempt.
## [1.1.8+1] - 09/05/2020
* Updated Gherkin library version to sort issue with JSON reporter throwing error when an exception is logged before any feature have run

287
README.md
View File

@ -10,7 +10,7 @@ This implementation of the Gherkin tries to follow as closely as possible other
Available as a Dart package https://pub.dartlang.org/packages/flutter_gherkin
```dart
``` dart
# Comment
Feature: Addition
@ -26,13 +26,12 @@ Available as a Dart package https://pub.dartlang.org/packages/flutter_gherkin
Then I end up with 2
```
NOTE: If you are using a Flutter branch other than the current stable version 1.12.x you will need to use the release candiate version of this library due to a breaking change with the way the flutter driver logs output.
## Table of Contents
<!-- TOC -->
- [Getting Started](#getting-started)
- [Configuration](#configuration)
* [Getting Started](#getting-started)
+ [Configuration](#configuration)
- [features](#features)
- [tagExpression](#tagexpression)
- [order](#order)
@ -42,39 +41,44 @@ NOTE: If you are using a Flutter branch other than the current stable version 1.
- [hooks](#hooks)
- [reporters](#reporters)
- [createWorld](#createworld)
- [logFlutterProcessOutput](#logFlutterProcessOutput)
- [flutterBuildTimeout](#flutterBuildTimeout)
- [exitAfterTestRun](#exitaftertestrun)
- [Flutter specific configuration options](#flutter-specific-configuration-options)
+ [Flutter specific configuration options](#flutter-specific-configuration-options)
- [restartAppBetweenScenarios](#restartappbetweenscenarios)
- [build](#build)
- [buildFlavor](#buildFlavor)
- [flutterBuildTimeout](#flutterBuildTimeout)
- [logFlutterProcessOutput](#logFlutterProcessOutput)
- [targetDeviceId](#targetDeviceId)
- [targetAppPath](#targetapppath)
- [Features Files](#features-files)
- [Steps Definitions](#steps-definitions)
- [runningAppProtocolEndpointUri](#runningAppProtocolEndpointUri)
- [onBeforeFlutterDriverConnect](#onBeforeFlutterDriverConnect)
- [onAfterFlutterDriverConnect](#onAfterFlutterDriverConnect)
- [flutterDriverMaxConnectionAttempts](#flutterDriverMaxConnectionAttempts)
- [flutterDriverReconnectionDelay](#flutterDriverReconnectionDelay)
* [Features Files](#features-files)
+ [Steps Definitions](#steps-definitions)
- [Given](#given)
- [Then](#then)
- [Step Timeout](#step-timeout)
- [Multiline Strings](#multiline-strings)
- [Data tables](#data-tables)
- [Well known step parameters](#well-known-step-parameters)
- [Pluralisation](#pluralisation)
- [Pluralization](#pluralization)
- [Custom Parameters](#custom-parameters)
- [World Context (per test scenario shared state)](#world-context-per-test-scenario-shared-state)
- [Assertions](#assertions)
- [Tags](#tags)
- [Languages](#languages)
- [Hooks](#hooks)
- [Attachements](#attachments)
- [Screenshot on step failure](#screenshot)
- [Reporting](#reporting)
- [Flutter](#flutter)
- [Restarting the app before each test](#restarting-the-app-before-each-test)
+ [Tags](#tags)
+ [Languages](#languages)
* [Hooks](#hooks)
* [Attachments](#attachments)
+ [Screenshot on step failure](#screenshot)
* [Reporting](#reporting)
* [Flutter](#flutter)
+ [Restarting the app before each test](#restarting-the-app-before-each-test)
- [Flutter World](#flutter-world)
- [Pre-defined Steps](#pre-defined-steps)
+ [Pre-defined Steps](#pre-defined-steps)
- [Flutter Driver Utilities](#flutter-driver-utilities)
- [Debugging](#debugging)
+ [Debugging](#debugging)
- [Debugging the app under test](#debugging-the-app-under-test)
<!-- /TOC -->
@ -87,7 +91,7 @@ See [example readme](example/README.md) for a quick start guide to running the e
The first step is to create a version of your app that has flutter driver enabled so that it can be automated. A good guide how to do this is show [here](https://flutter.io/cookbook/testing/integration-test-introduction/#4-instrument-the-app). However in short, create a folder called `test_driver` and within that create a file called `app.dart` and paste in the below code.
```dart
``` dart
import '../lib/main.dart';
import 'package:flutter/widgets.dart';
import 'package:flutter_driver/driver_extension.dart';
@ -107,9 +111,9 @@ All this code does is enable the Flutter driver extension which is required to b
To get started with BDD in Flutter the first step is to write a feature file and a test scenario within that.
First create a folder called `test_driver` (this is inline with the current integration test as we will need to use the Flutter driver to automate the app). Within the folder create a folder called `features`, then create a file called `counter.feature`.
First create a folder called `test_driver` (this is inline with the current integration test as we will need to use the Flutter driver to automate the app). Within the folder create a folder called `features` , then create a file called `counter.feature` .
```dart
``` dart
Feature: Counter
The counter should be incremented when the button is pressed.
@ -119,13 +123,13 @@ Feature: Counter
Then I expect the "counter" to be "10"
```
Now we have created a scenario we need to implement the steps within. Steps are just classes that extends from the base step definition class or any of its variations `Given`, `Then`, `When`, `And`, `But`.
Now we have created a scenario we need to implement the steps within. Steps are just classes that extends from the base step definition class or any of its variations `Given` , `Then` , `When` , `And` , `But` .
Granted the example is a little contrived but is serves to illustrate the process.
This library has a couple of built in step definitions for convenience. The first step uses the built in step, however the second step `When I tap the "increment" button 10 times` is a custom step and has to be implemented. To implement a step we have to create a simple step definition class.
```dart
``` dart
import 'package:flutter_driver/flutter_driver.dart';
import 'package:flutter_gherkin/flutter_gherkin.dart';
import 'package:gherkin/gherkin.dart';
@ -149,9 +153,9 @@ class TapButtonNTimesStep extends When2WithWorld<String, int, FlutterWorld> {
As you can see the class inherits from `When2WithWorld` and specifies the types of the two input parameters. The third type `FlutterWorld` is a special Flutter context object that allow access to the Flutter driver instance within the step. If you did not need this you could inherit from `When2` which does not type the world context object but still provides two input parameters.
The input parameters are retrieved via the pattern regex from well know parameter types `{string}` and `{int}` [explained below](#well-known-step-parameters). They are just special syntax to indicate you are expecting a string and an integer at those points in the step text. Therefore, when the step to execute is `When I tap the "increment" button 10 times` the parameters "increment" and 10 will be passed into the step as the correct types. Note that in the pattern you can use any regex capture group to indicate any input parameter. For example the regex ```RegExp(r"When I tap the {string} (button|icon) {int} times")``` indicates 3 parameters and would match to either of the below step text.
The input parameters are retrieved via the pattern regex from well know parameter types `{string}` and `{int}` [explained below](#well-known-step-parameters). They are just special syntax to indicate you are expecting a string and an integer at those points in the step text. Therefore, when the step to execute is `When I tap the "increment" button 10 times` the parameters "increment" and 10 will be passed into the step as the correct types. Note that in the pattern you can use any regex capture group to indicate any input parameter. For example the regex ` ` ` RegExp(r"When I tap the {string} (button|icon) {int} times") ` ` ` indicates 3 parameters and would match to either of the below step text.
```dart
``` dart
When I tap the "increment" button 10 times // passes 3 parameters "increment", "button" & 10
When I tap the "increment" icon 2 times // passes 3 parameters "increment", "icon" & 2
```
@ -160,7 +164,7 @@ It is worth noting that this library *does not* rely on mirrors (reflection) for
Now that we have a testable app, a feature file and a custom step definition we need to create a class that will call this library and actually run the tests. Create a file called `app_test.dart` and put the below code in.
```dart
``` dart
import 'dart:async';
import 'package:flutter_gherkin/flutter_gherkin.dart';
import 'package:gherkin/gherkin.dart';
@ -189,11 +193,11 @@ Future<void> main() {
}
```
This code simple creates a configuration object and calls this library which will then promptly parse your feature files and run the tests. The configuration file is important and explained in further detail below. However, all that is happening is a `Glob` is provide which specifies the path to one or more feature files, it sets the reporters to the `ProgressReporter` report which prints the result of scenarios and steps to the standard output (console). The `TestRunSummaryReporter` prints a summary of the run once all tests have been executed. Finally it specifies the path to the testable app created above `test_driver/app.dart`. This is important as it instructions the library which app to run the tests against.
This code simple creates a configuration object and calls this library which will then promptly parse your feature files and run the tests. The configuration file is important and explained in further detail below. However, all that is happening is a `Glob` is provide which specifies the path to one or more feature files, it sets the reporters to the `ProgressReporter` report which prints the result of scenarios and steps to the standard output (console). The `TestRunSummaryReporter` prints a summary of the run once all tests have been executed. Finally it specifies the path to the testable app created above `test_driver/app.dart` . This is important as it instructions the library which app to run the tests against.
Finally to actually run the tests run the below on the command line:
```bash
``` bash
dart test_driver/app_test.dart
```
@ -215,23 +219,23 @@ An iterable of `Glob` patterns that specify the location(s) of `*.feature` files
#### tagExpression
Defaults to `null`.
Defaults to `null` .
An infix boolean expression which defines the features and scenarios to run based of their tags. See [Tags](#tags).
#### order
Defaults to `ExecutionOrder.random`
Defaults to `ExecutionOrder.random`
The order by which scenarios will be run. Running an a random order may highlight any inter-test dependencies that should be fixed.
#### stepDefinitions
Defaults to `Iterable<StepDefinitionBase>`
Defaults to `Iterable<StepDefinitionBase>`
Place instances of any custom step definition classes `Given`, `Then`, `When`, `And`, `But` that match to any custom steps defined in your feature files.
Place instances of any custom step definition classes `Given` , `Then` , `When` , `And` , `But` that match to any custom steps defined in your feature files.
```dart
``` dart
import 'dart:async';
import 'package:flutter_gherkin/flutter_gherkin.dart';
import 'package:gherkin/gherkin.dart';
@ -254,13 +258,13 @@ Future<void> main() {
#### defaultLanguage
Defaults to `en`
Defaults to `en`
This specifies the default langauge the feature files are written in. See https://cucumber.io/docs/gherkin/reference/#overview for supported languages.
This specifies the default language the feature files are written in. See https://cucumber.io/docs/gherkin/reference/#overview for supported languages.
Note that this can be overriden in the feature itself by the use of a language block.
Note that this can be overridden in the feature itself by the use of a language block.
```
```
# language: de
Funktionalität: Calculator
Tests the addition of two numbers
@ -278,7 +282,7 @@ Funktionalität: Calculator
| 20.937 | -1.937 | 19 |
```
```
```
# language: fr
Fonctionnalité: Counter
The counter should be incremented when the button is pressed.
@ -293,11 +297,11 @@ Fonctionnalité: Counter
#### customStepParameterDefinitions
Defaults to `CustomParameter<dynamic>`.
Defaults to `CustomParameter<dynamic>` .
Place instances of any custom step parameters that you have defined. These will be matched up to steps when scenarios are run and their result passed to the executable step. See [Custom Parameters](#custom-parameters).
```dart
``` dart
import 'dart:async';
import 'package:flutter_gherkin/flutter_gherkin.dart';
import 'package:gherkin/gherkin.dart';
@ -327,9 +331,9 @@ Hooks are custom bits of code that can be run at certain points with the test ru
Attachment are pieces of data you can attach to a running scenario. This could be simple bits of textual data or even image like a screenshot. These attachments can then be used by reporters to provide more contextual information. For example when a step fails some contextual information could be attached to the scenario which is then used by a reporter to display why the step failed.
Attachments would typically be attached via a `Hook` for example `onAfterStep`.
Attachments would typically be attached via a `Hook` for example `onAfterStep` .
```
```
import 'package:gherkin/gherkin.dart';
class AttachScreenshotOnFailedStepHook extends Hook {
@ -345,12 +349,11 @@ class AttachScreenshotOnFailedStepHook extends Hook {
```
##### screenshot
To take a screenshot on a step failing you can used the pre-defined hook `AttachScreenshotOnFailedStepHook` and include it in the hook configuration of the tests config. This hook will take a screenshot and add it as an attachment to the scenerio. If the `JsonReporter` is being used the screenshot will be embedded in the report which can be used to generate a HTML report which will ultimately display the screenshot under the failed step.
To take a screenshot on a step failing you can used the pre-defined hook `AttachScreenshotOnFailedStepHook` and include it in the hook configuration of the tests config. This hook will take a screenshot and add it as an attachment to the scenario. If the `JsonReporter` is being used the screenshot will be embedded in the report which can be used to generate a HTML report which will ultimately display the screenshot under the failed step.
```
```
import 'dart:async';
import 'package:flutter_gherkin/flutter_gherkin.dart';
import 'package:gherkin/gherkin.dart';
@ -382,15 +385,15 @@ Future<void> main() {
Reporters are classes that are able to report on the status of the test run. This could be a simple as merely logging scenario result to the console. There are a number of built-in reporter:
- `StdoutReporter` : Logs all messages from the test run to the standard output (console).
- `ProgressReporter` : Logs the progress of the test run marking each step with a scenario as either passed, skipped or failed.
- `JsonReporter` - creates a JSON file with the results of the test run which can then be used by 'https://www.npmjs.com/package/cucumber-html-reporter.' to create a HTML report. You can pass in the file path of the json file to be created.
* `StdoutReporter` : Logs all messages from the test run to the standard output (console).
* `ProgressReporter` : Logs the progress of the test run marking each step with a scenario as either passed, skipped or failed.
* `JsonReporter` - creates a JSON file with the results of the test run which can then be used by 'https://www.npmjs.com/package/cucumber-html-reporter.' to create a HTML report. You can pass in the file path of the json file to be created.
You should provide at least one reporter in the configuration otherwise it'll be hard to know what is going on.
*Note*: Feel free to PR new reporters!
```dart
``` dart
import 'dart:async';
import 'package:glob/glob.dart';
import 'package:flutter_gherkin/flutter_gherkin.dart';
@ -413,11 +416,11 @@ Future<void> main() {
#### createWorld
Defaults to `null`.
Defaults to `null` .
While it is not recommended so share state between steps within the same scenario we all in fact live in the real world and thus at time may need to share certain information such as login credentials etc for future steps to use. The world context object is created once per scenario and then destroyed at the end of each scenario. This configuration property allows you to specify a custom `World` class to create which can then be accessed in your step classes.
```dart
``` dart
import 'dart:async';
import 'package:glob/glob.dart';
import 'package:flutter_gherkin/flutter_gherkin.dart';
@ -439,19 +442,39 @@ Future<void> main() {
#### logFlutterProcessOutput
Defaults to `false`
Defaults to `false`
If `true` the output from the flutter process is logged to the stdout / stderr streams. Useful when debugging app build or start failures
#### flutterBuildTimeout
Defaults to `90 seconds`
Defaults to `90 seconds`
Specifies the period of time to wait for the Flutter build to complete and the app to be installed and in a state to be tested. Slower machines may need longer than the default 90 seconds to complete this process.
#### onBeforeFlutterDriverConnect
An async method that is called before an attempt by Flutter driver to connect to the app under test
#### onAfterFlutterDriverConnect
An async method that is called after a successful attempt by Flutter driver to connect to the app under test
#### flutterDriverMaxConnectionAttempts
Defaults to `3`
Specifies the number of Flutter driver connection attempts to a running app before the test is aborted
#### flutterDriverReconnectionDelay
Defaults to `2 seconds`
Specifies the amount of time to wait after a failed Flutter driver connection attempt to the running app
#### exitAfterTestRun
Defaults to `true`
Defaults to `true`
True to exit the program after all tests have run. You may want to set this to false during debugging.
@ -461,49 +484,49 @@ The `FlutterTestConfiguration` will automatically create some default Flutter op
#### restartAppBetweenScenarios
Defaults to `true`.
Defaults to `true` .
To avoid tests starting on an app changed by a previous test it is suggested that the Flutter application under test be restarted between each scenario. While this will increase the execution time slightly it will limit tests failing because they run against an app changed by a previous test. Note in more complex application it may also be necessary to use the `AfterScenario` hook to reset the application to a base state a test can run on. Logging out for example if restarting an application will present a lock screen etc. This now performs a hot reload of the application which resets the state and drastically reduces the time to run the tests.
#### targetAppPath
Defaults to `lib/test_driver/app.dart`
Defaults to `lib/test_driver/app.dart`
This should point to the *testable* application that enables the Flutter driver extensions and thus is able to be automated. This application wil be started when the test run is started and restarted if the `restartAppBetweenScenarios` configuration property is set to true.
#### build
Defaults to `true`
Defaults to `true`
This optional argument lets you specify if the target application should be built prior to running the first test. This defaults to `true`
This optional argument lets you specify if the target application should be built prior to running the first test. This defaults to `true`
#### buildFlavor
Defaults to empty string
This optional argument lets you specify which flutter flavor you want to test against. Flutter's flavor has similar concept with `Android Build Variants` or `iOS Scheme Configuration`. This [flavoring flutter](https://flutter.dev/docs/deployment/flavors) documentation has complete guide on both flutter and android/ios side.
This optional argument lets you specify which flutter flavor you want to test against. Flutter's flavor has similar concept with `Android Build Variants` or `iOS Scheme Configuration` . This [flavoring flutter](https://flutter.dev/docs/deployment/flavors) documentation has complete guide on both flutter and android/ios side.
#### targetDeviceId
Defaults to empty string
This optional argument lets you specify device target id as `flutter run --device-id` command. To show list of connected devices, run `flutter devices`. If you only have one device connected, no need to provide this argument.
This optional argument lets you specify device target id as `flutter run --device-id` command. To show list of connected devices, run `flutter devices` . If you only have one device connected, no need to provide this argument.
#### runningAppProtocolEndpointUri
An observatory url that the test runner can connect to instead of creating a new running instance of the target application
The url takes the form of `http://127.0.0.1:51540/EM72VtRsUV0=/` and usually printed to stdout in the form `Connecting to service protocol: http://127.0.0.1:51540/EM72VtRsUV0=/`
The url takes the form of `http://127.0.0.1:51540/EM72VtRsUV0=/` and usually printed to stdout in the form `Connecting to service protocol: http://127.0.0.1:51540/EM72VtRsUV0=/`
You will have to add the `--verbose` flag to the command to start your flutter app to see this output and ensure `enableFlutterDriverExtension()` is called by the running app
## Features Files
### Steps Definitions
Step definitions are the coded representation of a textual step in a feature file. Each step starts with either `Given`, `Then`, `When`, `And` or `But`. It is worth noting that all steps are actually the same but semantically different. The keyword is not taken into account when matching a step. Therefore the two below steps are actually treated the same and will result in the same step definition being invoked.
Step definitions are the coded representation of a textual step in a feature file. Each step starts with either `Given` , `Then` , `When` , `And` or `But` . It is worth noting that all steps are actually the same but semantically different. The keyword is not taken into account when matching a step. Therefore the two below steps are actually treated the same and will result in the same step definition being invoked.
Note: Step definitions (in this implementation) are allowed up to 5 input parameters. If you find yourself needing more than this you might want to consider making your step more isolated or using a `Table` parameter.
```dart
``` dart
Given there are 6 kangaroos
Then there are 6 kangaroos
```
@ -514,15 +537,15 @@ However, the domain language you choose will influence what keyword works best i
`Given` steps are used to describe the initial state of a system. The execution of a `Given` step will usually put the system into well defined state.
To implement a `Given` step you can inherit from the ```Given``` class.
To implement a `Given` step you can inherit from the ` ` ` Given ` ` ` class.
```dart
``` dart
Given Bob has logged in
```
Would be implemented like so:
```dart
``` dart
import 'package:gherkin/gherkin.dart';
class GivenWellKnownUserIsLoggedIn extends Given1<String> {
@ -536,9 +559,9 @@ class GivenWellKnownUserIsLoggedIn extends Given1<String> {
}
```
If you need to have more than one Given in a block it is often best to use the additional keywords `And` or `But`.
If you need to have more than one Given in a block it is often best to use the additional keywords `And` or `But` .
```dart
``` dart
Given Bob has logged in
And opened the dashboard
```
@ -547,13 +570,13 @@ And opened the dashboard
`Then` steps are used to describe an expected outcome, or result. They would typically have an assertion in which can pass or fail.
```dart
``` dart
Then I expect 10 apples
```
Would be implemented like so:
```dart
``` dart
import 'package:gherkin/gherkin.dart';
class ThenExpectAppleCount extends Then1<int> {
@ -573,11 +596,11 @@ class ThenExpectAppleCount extends Then1<int> {
#### Step Timeout
By default a step will timeout if it exceed the `defaultTimeout` parameter in the configuration file. In some cases you want have a step that is longer or shorter running and in the case you can optionally proved a custom timeout to that step. To do this pass in a `Duration` object in the step's call to `super`.
By default a step will timeout if it exceed the `defaultTimeout` parameter in the configuration file. In some cases you want have a step that is longer or shorter running and in the case you can optionally proved a custom timeout to that step. To do this pass in a `Duration` object in the step's call to `super` .
For example, the below sets the step's timeout to 10 seconds.
```dart
``` dart
import 'package:flutter_driver/flutter_driver.dart';
import 'package:gherkin/gherkin.dart';
import 'package:flutter_gherkin/flutter_gherkin.dart';
@ -601,11 +624,11 @@ class TapButtonNTimesStep extends When2WithWorld<String, int, FlutterWorld> {
#### Multiline Strings
Multiline strings can follow a step and will be give to the step it proceeds as the final argument. To denote a multiline string the pre and postfix can either be third double or single quotes `""" ... """` or `''' ... '''`.
Multiline strings can follow a step and will be give to the step it proceeds as the final argument. To denote a multiline string the pre and postfix can either be third double or single quotes `""" ... """` or `''' ... '''` .
For example:
```dart
``` dart
Given I provide the following "review" comment
"""
Some long review comment.
@ -622,7 +645,7 @@ Maybe even include some numbers
The matching step definition would then be:
```dart
``` dart
import 'package:gherkin/gherkin.dart';
class GivenIProvideAComment extends Given2<String, String> {
@ -639,14 +662,14 @@ class GivenIProvideAComment extends Given2<String, String> {
#### Data tables
```dart
``` dart
import 'package:gherkin/gherkin.dart';
/// This step expects a multiline string proceeding it
///
/// For example:
///
/// `Given I add the users`
/// `Given I add the users`
/// | Firstname | Surname | Age | Gender |
/// | Woody | Johnson | 28 | Male |
/// | Edith | Summers | 23 | Female |
@ -663,7 +686,7 @@ class GivenIAddTheUsers extends Given1<Table> {
final columns = dataTable.asMap();
final personOne = columns.elementAt(0);
final personOneName = personOne["Firstname"];
print('Name of first user: `$personOneName`');
print('Name of first user: `$personOneName` ');
}
@override
@ -679,18 +702,18 @@ In most scenarios theses parameters will be enough for you to write quite advanc
| Parameter Name | Description | Aliases | Type | Example |
| -------------- | --------------------------------------------- | ------------------------------ | ------ | ------------------------------------------------------------------- |
| {word} | Matches a single word surrounded by a quotes | {word}, {Word} | String | `Given I eat a {word}` would match `Given I eat a "worm"` |
| {word} | Matches a single word surrounded by a quotes | {word}, {Word} | String | `Given I eat a {word}` would match `Given I eat a "worm"` |
| {string} | Matches one more words surrounded by a quotes | {string}, {String} | String | `Given I eat a {string}` would match `Given I eat a "can of worms"` |
| {int} | Matches an integer | {int}, {Int} | int | `Given I see {int} worm(s)` would match `Given I see 6 worms` |
| {num} | Matches an number | {num}, {Num}, {float}, {Float} | num | `Given I see {num} worm(s)` would match `Given I see 0.75 worms` |
| {int} | Matches an integer | {int}, {Int} | int | `Given I see {int} worm(s)` would match `Given I see 6 worms` |
| {num} | Matches an number | {num}, {Num}, {float}, {Float} | num | `Given I see {num} worm(s)` would match `Given I see 0.75 worms` |
Note that you can combine there well known parameters in any step. For example `Given I {word} {int} worm(s)` would match `Given I "see" 6 worms` and also match `Given I "eat" 1 worm`
Note that you can combine there well known parameters in any step. For example `Given I {word} {int} worm(s)` would match `Given I "see" 6 worms` and also match `Given I "eat" 1 worm`
#### Pluralisation
#### Pluralization
As the aim of a feature is to convey human readable tests it is often desirable to optionally have some word pluaralised so you can use the special pluralisation syntax to do simple pluralisation of some words in your step definition. For example:
As the aim of a feature is to convey human readable tests it is often desirable to optionally have some word pluralized so you can use the special pluralization syntax to do simple pluralization of some words in your step definition. For example:
The step string `Given I see {int} worm(s)` has the pluralisation syntax on the word "worm" and thus would be matched to both `Given I see 1 worm` and `Given I see 4 worms`.
The step string `Given I see {int} worm(s)` has the pluralization syntax on the word "worm" and thus would be matched to both `Given I see 1 worm` and `Given I see 4 worms` .
#### Custom Parameters
@ -698,7 +721,7 @@ While the well know step parameter will be sufficient in most cases there are ti
The below custom parameter defines a regex that matches the words "red", "green" or "blue". The matches word is passed into the function which is then able to convert the string into a Color object. The name of the custom parameter is used to identity the parameter within the step text. In the below example the word "colour" is used. This is combined with the pre / post prefixes (which default to "{" and "}") to match to the custom parameter.
```dart
``` dart
import 'package:gherkin/gherkin.dart';
enum Colour { red, green, blue }
@ -720,7 +743,7 @@ class ColourParameter extends CustomParameter<Colour> {
The step definition would then use this custom parameter like so:
```dart
``` dart
import 'package:gherkin/gherkin.dart';
import 'colour_parameter.dart';
@ -735,7 +758,7 @@ class GivenIPickAColour extends Given1<Colour> {
}
```
This customer parameter would be used like this: `Given I pick the colour red`. When the step is invoked the word "red" would matched and passed to the custom parameter to convert it into a `Colour` enum which is then finally passed to the step definition code as a `Colour` object.
This customer parameter would be used like this: `Given I pick the colour red` . When the step is invoked the word "red" would matched and passed to the custom parameter to convert it into a `Colour` enum which is then finally passed to the step definition code as a `Colour` object.
#### World Context (per test scenario shared state)
@ -743,21 +766,21 @@ This customer parameter would be used like this: `Given I pick the colour red`.
### Tags
Tags are a great way of organising your features and marking them with filterable information. Tags can be uses to filter the scenarios that are run. For instance you might have a set of smoke tests to run on every check-in as the full test suite is only ran once a day. You could also use an `@ignore` or `@todo` tag to ignore certain scenarios that might not be ready to run yet.
Tags are a great way of organizing your features and marking them with filterable information. Tags can be uses to filter the scenarios that are run. For instance you might have a set of smoke tests to run on every check-in as the full test suite is only ran once a day. You could also use an `@ignore` or `@todo` tag to ignore certain scenarios that might not be ready to run yet.
You can filter the scenarios by providing a tag expression to your configuration file. Tag expression are simple infix expressions such as:
`@smoke`
`@smoke`
`@smoke and @perf`
`@smoke and @perf`
`@billing or @onboarding`
`@billing or @onboarding`
`@smoke and not @ignore`
`@smoke and not @ignore`
You can even us brackets to ensure the order of precedence
`@smoke and not (@ignore or @todo)`
`@smoke and not (@ignore or @todo)`
You can use the usual boolean statement "and", "or", "not"
@ -765,13 +788,13 @@ Also see <https://docs.cucumber.io/cucumber/api/#tags>
### Languages
In order to allow features to be written in a number of languages, you can now write the keywords in languages other than English. To improve readability and flow, some languages may have more than one translation for any given keyword. See https://cucumber.io/docs/gherkin/reference/#overview for a list of supported langauges.
In order to allow features to be written in a number of languages, you can now write the keywords in languages other than English. To improve readability and flow, some languages may have more than one translation for any given keyword. See https://cucumber.io/docs/gherkin/reference/#overview for a list of supported languages.
You can set the default language of feature files in your project via the configuration setting see [defaultLanguage](#defaultLanguage)
For example these two features are the same the keywords are just written in different languages. Note the ```# language: de``` on the second feature. English is the default language.
For example these two features are the same the keywords are just written in different languages. Note the ` ` ` # language: de ` ` ` on the second feature. English is the default language.
```
```
Feature: Calculator
Tests the addition of two numbers
@ -789,7 +812,7 @@ Feature: Calculator
```
```
```
# language: de
Funktionalität: Calculator
Tests the addition of two numbers
@ -814,14 +837,14 @@ Please note the language data is take and attributed to the cucumber project htt
A hook is a point in the execution that custom code can be run. Hooks can be run at the below points in the test run.
- Before any tests run
- After all the tests have run
- Before each scenario
- After each scenario
* Before any tests run
* After all the tests have run
* Before each scenario
* After each scenario
To create a hook is easy. Just inherit from `Hook` and override the method(s) that signifies the point in the process you want to run code at. Note that not all methods need to be override, just the points at which you want to run custom code.
```dart
``` dart
import 'package:gherkin/gherkin.dart';
class HookExample extends Hook {
@ -860,7 +883,7 @@ class HookExample extends Hook {
Finally ensure the hook is added to the hook collection in your configuration file.
```dart
``` dart
import 'dart:async';
import 'package:flutter_gherkin/flutter_gherkin.dart';
import 'package:gherkin/gherkin.dart';
@ -887,25 +910,25 @@ Future<void> main() {
A reporter is a class that is able to report on the progress of the test run. In it simplest form it could just print messages to the console or be used to tell a build server such as TeamCity of the progress of the test run. The library has a number of built in reporters.
- `StdoutReporter` - prints all messages from the test run to the console.
- `ProgressReporter` - prints the result of each scenario and step to the console - colours the output.
- `TestRunSummaryReporter` - prints the results and duration of the test run once the run has completed - colours the output.
- `JsonReporter` - creates a JSON file with the results of the test run which can then be used by 'https://www.npmjs.com/package/cucumber-html-reporter.' to create a HTML report. You can pass in the file path of the json file to be created.
- `FlutterDriverReporter` - prints the output from Flutter Driver. Flutter driver logs all messages to the stderr stream by default so most CI servers would mark the process as failed if anything is logged to the stderr stream (even if the Flutter driver logs are only info messages). This reporter ensures the log messages are output to the most appropiate stream depending on their log level.
* `StdoutReporter` - prints all messages from the test run to the console.
* `ProgressReporter` - prints the result of each scenario and step to the console - colours the output.
* `TestRunSummaryReporter` - prints the results and duration of the test run once the run has completed - colours the output.
* `JsonReporter` - creates a JSON file with the results of the test run which can then be used by 'https://www.npmjs.com/package/cucumber-html-reporter.' to create a HTML report. You can pass in the file path of the json file to be created.
* `FlutterDriverReporter` - prints the output from Flutter Driver. Flutter driver logs all messages to the stderr stream by default so most CI servers would mark the process as failed if anything is logged to the stderr stream (even if the Flutter driver logs are only info messages). This reporter ensures the log messages are output to the most appropriate stream depending on their log level.
You can create your own custom reporter by inheriting from the base `Reporter` class and overriding the one or many of the methods to direct the output message. The `Reporter` defines the following methods that can be overridden. All methods must return a `Future<void>` and can be async.
- `onTestRunStarted`
- `onTestRunFinished`
- `onFeatureStarted`
- `onFeatureFinished`
- `onScenarioStarted`
- `onScenarioFinished`
- `onStepStarted`
- `onStepFinished`
- `onException`
- `message`
- `dispose`
* `onTestRunStarted`
* `onTestRunFinished`
* `onFeatureStarted`
* `onFeatureFinished`
* `onScenarioStarted`
* `onScenarioFinished`
* `onStepStarted`
* `onStepFinished`
* `onException`
* `message`
* `dispose`
Once you have created your custom reporter don't forget to add it to the `reporters` configuration file property.
@ -927,12 +950,12 @@ For convenience the library defines a number of pre-defined steps so you can get
| Step Text | Description | Examples |
| ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- |
| I tap the {string} [button\|element\|label\|icon\|field\|text\|widget] | Taps the element with the provided key ( given by the first input parameter) | `When I tap the "login" button`, `Then I tap the "save" icon` |
| I fill the {string} field with {string} | Fills the element with the provided key with the given value (given by the second input parameter) | `When I fill the "email" field with "someone@gmail.com"` |
| I expect the {string} to be {string} | Asserts that the element with the given key has the given string value | `Then I expect the "cost" to be "£10.95"` |
| I (open\|close) the drawer | Opens or closes the application default drawer | `When I open the drawer`, `And I close the drawer`|
| I expect the [button\|element\|label\|icon\|field\|text\|widget] {string} to be present within {int} second(s) | Expects a widget with the given key to be present within n secondss | `Then I expect the widget 'notification' to be present within 10 seconds`, `Then I expect the icon 'notification' to be present within 1 second` |
| I pause for {int} seconds | Pauses the test execution for the given seconds. Only use in debug scenarios or to inspect the state of the app | `Then I pause for 20 seconds` |
| I tap the {string} [button\|element\|label\|icon\|field\|text\|widget] | Taps the element with the provided key ( given by the first input parameter) | `When I tap the "login" button` , `Then I tap the "save" icon` |
| I fill the {string} field with {string} | Fills the element with the provided key with the given value (given by the second input parameter) | `When I fill the "email" field with "someone@gmail.com"` |
| I expect the {string} to be {string} | Asserts that the element with the given key has the given string value | `Then I expect the "cost" to be "£10.95"` |
| I (open\|close) the drawer | Opens or closes the application default drawer | `When I open the drawer` , `And I close the drawer` |
| I expect the [button\|element\|label\|icon\|field\|text\|widget] {string} to be present within {int} second(s) | Expects a widget with the given key to be present within n seconds | `Then I expect the widget 'notification' to be present within 10 seconds` , `Then I expect the icon 'notification' to be present within 1 second` |
| I pause for {int} seconds | Pauses the test execution for the given seconds. Only use in debug scenarios or to inspect the state of the app | `Then I pause for 20 seconds` |
| I restart the app | Restarts the app under test | `Then I restart the app` |
| I tap the back button | Taps the page default back button widget | `Then I tap the back button` |
@ -944,7 +967,7 @@ For convenience the library provides a static `FlutterDriverUtils` class that ab
In VSCode simply add add this block to your launch.json file (if you testable app is called `app_test.dart` and within the `test_driver` folder, if not replace that with the correct file path). Don't forget to put a break point somewhere!
```json
``` json
{
"name": "Debug Features Tests",
"request": "launch",
@ -956,7 +979,7 @@ In VSCode simply add add this block to your launch.json file (if you testable ap
After which the file will most likely look like this
```json
``` json
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
@ -985,4 +1008,4 @@ Setting the configuration property `runningAppProtocolEndpointUri` to the servic
NOTE: ensure the app you are trying to connect to calls `enableFlutterDriverExtension()` when it starts up otherwise the Flutter Driver will not be able to connect to it.
Also ensure that the `--verbose` flag is set when starting the app to test, this will then log the service protocol endpoint out to the console which is the uri you will need to set this property to. It usually takes the form of `Connecting to service protocol: http://127.0.0.1:51540/EM72VtRsUV0=/` so set the `runningAppProtocolEndpointUri` to `http://127.0.0.1:51540/EM72VtRsUV0=/` and then start the tests.
Also ensure that the `--verbose` flag is set when starting the app to test, this will then log the service protocol endpoint out to the console which is the uri you will need to set this property to. It usually takes the form of `Connecting to service protocol: http://127.0.0.1:51540/EM72VtRsUV0=/` so set the `runningAppProtocolEndpointUri` to `http://127.0.0.1:51540/EM72VtRsUV0=/` and then start the tests.

View File

@ -36,7 +36,7 @@ Future<void> main() {
..targetAppPath = 'test_driver/app.dart'
// ..buildFlavor = "staging" // uncomment when using build flavor and check android/ios flavor setup see android file android\app\build.gradle
// ..targetDeviceId = "all" // uncomment to run tests on all connected devices or set specific device target id
// ..tagExpression = "@smoke" // uncomment to see an example of running scenarios based on tag expressions
// ..tagExpression = '@smoke' // uncomment to see an example of running scenarios based on tag expressions
// ..logFlutterProcessOutput = true // uncomment to see command invoked to start the flutter test app
// ..verboseFlutterProcessLogs = true // uncomment to see the verbose output from the Flutter process
// ..flutterBuildTimeout = Duration(minutes: 3) // uncomment to change the default period that flutter is expected to build and start the app within

File diff suppressed because one or more lines are too long

View File

@ -1,3 +1,4 @@
import 'dart:async';
import 'dart:io';
import 'package:flutter_gherkin/flutter_gherkin.dart';
import 'package:flutter_gherkin/src/flutter/flutter_world.dart';
@ -71,13 +72,42 @@ class FlutterTestConfiguration extends TestConfiguration {
/// You will have to add the `--verbose` flag to the command to start your flutter app to see this output and ensure `enableFlutterDriverExtension()` is called by the running app
String runningAppProtocolEndpointUri;
/// Called before any attempt to connect Flutter driver to the running application, Depending on your configuration this
/// method will be called before each scenario is run.
Future<void> Function() onBeforeFlutterDriverConnect;
/// Called after the successful connection of Flutter driver to the running application. Depending on your configuration this
/// method will be called on each new connection usually before each scenario is run.
Future<void> Function(FlutterDriver driver) onAfterFlutterDriverConnect;
void setObservatoryDebuggerUri(String uri) => _observatoryDebuggerUri = uri;
Future<FlutterDriver> createFlutterDriver([String dartVmServiceUrl]) async {
final completer = Completer<FlutterDriver>();
dartVmServiceUrl = (dartVmServiceUrl ?? _observatoryDebuggerUri) ??
Platform.environment['VM_SERVICE_URL'];
return await _attemptDriverConnection(dartVmServiceUrl, 1, 3);
await runZonedGuarded(
() async {
if (onBeforeFlutterDriverConnect != null) {
await onBeforeFlutterDriverConnect();
}
final driver = await _attemptDriverConnection(dartVmServiceUrl, 1, 3);
if (onAfterFlutterDriverConnect != null) {
await onAfterFlutterDriverConnect(driver);
}
completer.complete(driver);
},
(Object e, StackTrace st) {
if (e is DriverError) {
completer.completeError(e, st);
}
},
);
return completer.future;
}
Future<FlutterWorld> createFlutterWorld(
@ -93,6 +123,7 @@ class FlutterTestConfiguration extends TestConfiguration {
? flutterConfig.runningAppProtocolEndpointUri
: null,
);
world.setFlutterDriver(driver);
return world;
@ -122,7 +153,7 @@ class FlutterTestConfiguration extends TestConfiguration {
WhenPauseStep(),
WhenFillFieldStep(),
ThenExpectWidgetToBePresent(),
RestartAppStep()
RestartAppStep(),
]);
}
@ -131,24 +162,26 @@ class FlutterTestConfiguration extends TestConfiguration {
int attempt,
int maxAttempts,
) async {
try {
return await FlutterDriver.connect(
dartVmServiceUrl: dartVmServiceUrl,
);
} catch (e) {
if (attempt > maxAttempts) {
rethrow;
} else {
print(e);
await Future<void>.delayed(flutterDriverReconnectionDelay);
return await FlutterDriver.connect(
dartVmServiceUrl: dartVmServiceUrl,
).catchError(
(e, st) async {
if (attempt > maxAttempts) {
throw e;
} else {
print(
'Fluter driver error connecting to application at `$dartVmServiceUrl`, retrying after delay of $flutterDriverReconnectionDelay',
);
await Future<void>.delayed(flutterDriverReconnectionDelay);
return _attemptDriverConnection(
dartVmServiceUrl,
attempt + 1,
maxAttempts,
);
}
}
return _attemptDriverConnection(
dartVmServiceUrl,
attempt + 1,
maxAttempts,
);
}
},
);
}
void _ensureCorrectConfiguration() {

View File

@ -118,7 +118,7 @@ packages:
name: gherkin
url: "https://pub.dartlang.org"
source: hosted
version: "1.1.8"
version: "1.1.8+1"
glob:
dependency: "direct main"
description:

View File

@ -1,6 +1,6 @@
name: flutter_gherkin
description: A Gherkin / Cucumber parser and test runner for Dart and Flutter
version: 1.1.8+1
version: 1.1.8+2
homepage: https://github.com/jonsamwell/flutter_gherkin
environment:
@ -16,7 +16,7 @@ dependencies:
sdk: flutter
glob: ^1.1.7
meta: ">=1.1.6 <2.0.0"
gherkin: ^1.1.8
gherkin: ^1.1.8+1
# gherkin:
# path: ../dart_gherkin