Testral

Sign in with Email and Password.tl

passed 7 passed Duration 53.8s Avg per step 7.7s Started 2026-07-26 16:30:03 UTC Finished 2026-07-26 16:31:02 UTC

Environment

Operating SystemAndroid
Device Model
Appcom.testral.sample
App Version
Device IDCADD0EBA-4DE6-4027-95CC-9EB192B2C196

Steps

passed CLEAR_APP_DATA com.testral.sample 6.6s

Timing breakdown

  • Wait for UI settle (after)4.3s (64%)
passed launch the app 3.5s

Timing breakdown

  • Wait for UI settle (after)2.1s (60%)
passed tap "Got it" 14.0s
Device action
tap "Got it" @ 320 497

Timing breakdown

  • Before screenshot62ms (0%)
  • Wait for target in UI368ms (3%)
  • UI hierarchy dump245ms (2%)
  • Device tap9.3s (66%)
  • Wait for UI settle (after)430ms (3%)
  • Extra wait (test settings)500ms (4%)
  • After screenshot2.8s (20%)
  • Verify (heuristic — skipped LLM)0ms

Heuristic verify detected a screen change and skipped the LLM call.

Dialog or permission prompt dismissed — screen changed after tap.

Action screenshot
After screenshot
passed tap "Email & Password button" 6.3s
Device action
tap "Email & Password" @ 150 377

Timing breakdown

  • Wait for UI settle (before)745ms (12%)
  • Before screenshot60ms (1%)
  • Wait for target in UI384ms (6%)
  • Hierarchy miss268ms (4%)
  • Device tap1.6s (25%)
  • Wait for UI settle (after)1.3s (20%)
  • Extra wait (test settings)500ms (8%)
  • After screenshot62ms (1%)
  • Verify (LLM vision + bounds)1.1s (18%)

Hierarchy miss — one LLM vision call verified the step.

Located: "Email & Password button"

The UI changed from the 'Setup' screen to a 'Login' screen with fields for email and password, which is the expected outcome after tapping 'Email & Password button' (represented as 'Login' with 'Email, password, and confirmation' description in the before screenshot).

LLM (1)

verify google/gemini-2.5-flash
System prompt
You are a mobile test verifier. Given before/after screenshots and the expected outcome, return JSON:
{ "passed": boolean, "reasoning": string }
Return ONLY valid JSON.

Rules:
- For tap steps on buttons, dialogs, permission prompts, or popups: PASS when the dialog is dismissed or the app navigates to a new screen. A different after screenshot usually means success, not failure.
- Idempotent taps: if before and after look the same AND the target already appears selected/active/highlighted (tab, chip, segment, toggle already on), PASS — the desired state was already achieved. Do not fail merely because the UI did not change.
- Only FAIL a tap when the UI is unchanged AND the target still looks unselected/untouched (missed tap), or a clear error is shown.
- For other steps, verify the described expected outcome against the after screenshot.
User prompt
Tap step: "tap "Email & Password button""
Target tapped: Email & Password button
Before and after screenshots are attached.

Pass if ANY of these hold:
- A dialog/permission/system prompt related to the target was dismissed, OR
- The UI changed in a way that matches tapping "Email & Password button" (e.g. that control activated, its screen opened, or the expected in-app destination appeared), OR
- Before and after look the same AND "Email & Password button" already appears selected/active/highlighted (idempotent tap — already on the desired tab/chip/option)

Fail if:
- Before and after look essentially the same AND the target still looks unselected/untouched (tap missed)
- The app left the product under test for an unrelated system screen (App Store, iTunes, Settings, SpringBoard) unless the tap target was explicitly that destination
- A visible error state appeared
- The after screen has nothing to do with "Email & Password button"

The user message also included screenshot image(s) as vision inputs. Sent: before: 471×1024 · 35 KB · jpeg; after: 471×1024 · 21 KB · jpeg.

Model output
```json
{
  "passed": true,
  "reasoning": "The UI changed from the 'Setup' screen to a 'Login' screen with fields for email and password, which is the expected outcome after tapping 'Email & Password button' (represented as 'Login' with 'Email, password, and confirmation' description in the before screenshot)."
}
```
Action screenshot
After screenshot
passed type "test@example.com" into "Email text input" 11.6s
Device action
tap "Email text input" @ 201 275

Timing breakdown

  • Wait for target in UI174ms (2%)
  • UI hierarchy dump117ms (1%)
  • Locate target (L3 · google/gemini-2.5-flash)1.6s (13%)
  • Device tap1.2s (10%)
  • Type text6.7s (58%)
  • Extra wait (test settings)500ms (4%)
  • After screenshot59ms (1%)
  • Verify (heuristic — skipped LLM)0ms

Heuristic verify detected a screen change and skipped the LLM call.

Located via vision.

LLM (1)

locate google/gemini-2.5-flash
System prompt
You are a mobile UI test assistant. Given a screenshot and a target description, return JSON with:
{ "x": number, "y": number, "width": number, "height": number, "confidence": number, "reasoning": string }
Return ONLY valid JSON.


Rules:
- Coordinates use a 0–1000 grid over the provided image (status bar may already be cropped): (0,0) = top-left, (1000,1000) = bottom-right. Do NOT return raw pixel coordinates.
- Target interactive in-app UI only (buttons, links, inputs, list items, chips).
- NEVER target the system status bar (time, battery, signal, wifi icons) or navigation bar.
- App header actions like "Go to App", "Skip", "Done", "Close", or "Send" are below the status bar in the nav/toolbar — never in the system status-bar icon strip, and NEVER a software-keyboard key.
- Icon-only toolbar actions (circular send/share arrows, X close) still count as those labels — box the icon, not the message field or keyboard.
- When a container/location is given (e.g. "chips on the top", "in the header"), you MUST pick the instance in that region. If the same label also appears in a list/favourites/feed lower on screen, ignore those duplicates.
- When UiAutomator hierarchy candidates are provided, strongly prefer matching one of those bounds unless the screenshot clearly contradicts them — but still respect the container/location constraint.
- The bounding box must cover the visible tappable label/control, not nearby chrome.
- Ordinal list targets ("first card", "second row", "last result"): box exactly ONE list row/card — never a section header plus multiple rows, and never a tall stack of results. Prefer a tight row-height rectangle.
User prompt
Find the bounding box of the UI element: Email text input
Screenshot size: 1206x2512px (status bar cropped; may be downscaled for the model). Use 0–1000 grid coords relative to the attached image.

This image does not include the system status bar. Target the visible in-app control only.

The user message also included a screenshot as a vision input. Sent: screenshot: 492×1024 · 21 KB · jpeg.

Model output
```json
{
  "x": 50,
  "y": 250,
  "width": 900,
  "height": 70,
  "confidence": 0.9,
  "reasoning": "The target is 'Email text input'. The screenshot shows a text input field with the placeholder text 'Email'. This field is located below the 'Sign in with email and password' text and above the 'Password' text input field."
}
```
Action screenshot
After screenshot
passed type "password123" into "Password text input" 11.5s
Device action
tap "Password text input" @ 201 331

Timing breakdown

  • Wait for target in UI207ms (2%)
  • UI hierarchy dump149ms (1%)
  • Locate target (L3 · google/gemini-2.5-flash)1.8s (15%)
  • Device tap1.1s (10%)
  • Type text6.5s (56%)
  • Extra wait (test settings)500ms (4%)
  • After screenshot62ms (1%)
  • Verify (heuristic — skipped LLM)0ms

Heuristic verify detected a screen change and skipped the LLM call.

Password field shows 11 masked characters (expected 11)

LLM (1)

locate google/gemini-2.5-flash
System prompt
You are a mobile UI test assistant. Given a screenshot and a target description, return JSON with:
{ "x": number, "y": number, "width": number, "height": number, "confidence": number, "reasoning": string }
Return ONLY valid JSON.


Rules:
- Coordinates use a 0–1000 grid over the provided image (status bar may already be cropped): (0,0) = top-left, (1000,1000) = bottom-right. Do NOT return raw pixel coordinates.
- Target interactive in-app UI only (buttons, links, inputs, list items, chips).
- NEVER target the system status bar (time, battery, signal, wifi icons) or navigation bar.
- App header actions like "Go to App", "Skip", "Done", "Close", or "Send" are below the status bar in the nav/toolbar — never in the system status-bar icon strip, and NEVER a software-keyboard key.
- Icon-only toolbar actions (circular send/share arrows, X close) still count as those labels — box the icon, not the message field or keyboard.
- When a container/location is given (e.g. "chips on the top", "in the header"), you MUST pick the instance in that region. If the same label also appears in a list/favourites/feed lower on screen, ignore those duplicates.
- When UiAutomator hierarchy candidates are provided, strongly prefer matching one of those bounds unless the screenshot clearly contradicts them — but still respect the container/location constraint.
- The bounding box must cover the visible tappable label/control, not nearby chrome.
- Ordinal list targets ("first card", "second row", "last result"): box exactly ONE list row/card — never a section header plus multiple rows, and never a tall stack of results. Prefer a tight row-height rectangle.
User prompt
Find the bounding box of the UI element: Password text input
Screenshot size: 1206x2512px (status bar cropped; may be downscaled for the model). Use 0–1000 grid coords relative to the attached image.

This image does not include the system status bar. Target the visible in-app control only.

The user message also included a screenshot as a vision input. Sent: screenshot: 492×1024 · 22 KB · jpeg.

Model output
```json
{
  "x": 56,
  "y": 308,
  "width": 888,
  "height": 88,
  "confidence": 0.9,
  "reasoning": "The target is the 'Password' text input field. Based on the screenshot, the 'Password' text is visible within a rectangular input field. The coordinates provided correspond to this input field."
}
```
Action screenshot
After screenshot
passed verify "Sign in button" is visible 223ms

Timing breakdown

  • Before screenshot75ms (34%)
  • UI hierarchy dump148ms (66%)
  • Verify (hierarchy match — skipped LLM)0ms

Assertion resolved from the UI hierarchy — no LLM call.

Matched: "Sign in button"

Hierarchy match for "Sign in button" — skipped LLM

Action screenshot

Test commands

Copy this script to reproduce the test in Testral.

bundleId com.testral.sample
CLEAR_APP_DATA com.testral.sample
name "Sign in with Email and Password"
launch the app
tap "Got it"
tap "Email & Password button"
type "test@example.com" into "Email text input"
type "password123" into "Password text input"
verify "Sign in button" is visible