Bug 92231 - [Qt][WK2] REGRESSION(r123583): Removing a test made 88 tests flakey
Summary: [Qt][WK2] REGRESSION(r123583): Removing a test made 88 tests flakey
Status: RESOLVED DUPLICATE of bug 101353
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: All All
: P1 Critical
Assignee: Csaba Osztrogonác
URL:
Keywords: Qt, QtTriaged
Depends on:
Blocks: 79666 92111 96861
  Show dependency treegraph
 
Reported: 2012-07-25 02:38 PDT by Csaba Osztrogonác
Modified: 2012-11-06 22:40 PST (History)
7 users (show)

See Also:


Attachments
Patch (1.38 KB, patch)
2012-07-25 09:48 PDT, Csaba Osztrogonác
hausmann: review-
gyuyoung.kim: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Csaba Osztrogonác 2012-07-25 02:38:00 PDT
I have no idead how is it possible, but it must be a serious bug somewhere in Qt-WK2 port.

r123582 -  22 flakes - http://build.webkit.sed.hu/builders/x86-64%20Linux%20Qt%20Release%20WebKit2%20%28Amazon%20EC2%29/builds/6482
r123584 - 110 flakes - http://build.webkit.sed.hu/builders/x86-64%20Linux%20Qt%20Release%20WebKit2%20%28Amazon%20EC2%29/builds/6483
Comment 1 Zan Dobersek 2012-07-25 03:49:05 PDT
From little experience gained from poking around the Gtk's implementation, I think the EventSenderProxy lacks a method that would reset all the members to their default values before each test is run. Currently the EventSenderProxy is created at TestController initialization and just reused for every test without being reset.

Looking through the Qt's implementation I can see a problem occurring if two subsequent tests happen to click with the same button in the exact same location with not enough timespan between them - this would generate a double click in the second test instead of a simple click. Other similar problems can occur due to no resetting of the EventSenderProxy state.

I'm not saying that's exactly the problem nor do I have any evidence of this actually happening and causing increase in flaky tests. It just seems to me this might be the most possible explanation to the problem due to the flakiness and the fact this started occurring after a test utilizing the EventSender was removed from the test run.
Comment 2 Csaba Osztrogonác 2012-07-25 09:48:51 PDT
Created attachment 154378 [details]
Patch

Thanks for the idea, I tried to reinstantiate EventSenderProxy between each tests, and it works for me. Maybe there is simpler solution, but I don't know ...
Comment 3 Gyuyoung Kim 2012-07-25 12:17:57 PDT
Comment on attachment 154378 [details]
Patch

Attachment 154378 [details] did not pass efl-ews (efl):
Output: http://queues.webkit.org/results/13339346
Comment 4 Gustavo Noronha (kov) 2012-07-25 14:53:33 PDT
Comment on attachment 154378 [details]
Patch

Attachment 154378 [details] did not pass gtk-ews (gtk):
Output: http://queues.webkit.org/results/13339395
Comment 5 Simon Hausmann 2012-07-26 08:04:04 PDT
Comment on attachment 154378 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=154378&action=review

> Tools/WebKitTestRunner/TestController.cpp:432
> +    if (m_eventSenderProxy)
> +        delete m_eventSenderProxy;

You don't need to check m_eventSenderProxy for being non-null before deleting it. delete 0 is fine :)

However I don't think this approach is quite right. It _seems_ the cause is the event queues in the event sender proxy, which is something only non-mac ports "suffer" from.

Either our approach of leapForward should be imlemented differently (can we specify time stamps in the events like mac?) or perhaps we are missing some sort of reset() call here in resetStateToConsistentValues() that clears the queue and has a comment explaining why. Deleting and re-creating the event sender proxy is just hiding the real problem IMHO.
Comment 6 Csaba Osztrogonác 2012-09-15 08:01:06 PDT
There are many notifydone-timeout test (and pass after retry) because of 
this bug. Unfortunately https://trac.webkit.org/changeset/128395 increased
the notifydone timeout from 6 seconds to 30 seconds. It slowed down the
testing time from 15.5 minutes to 23.5 minutes.

I'm back to fix this bug, runtime is good for me, but there are still
4-5 fails (not flakey, fail), so it seems reseting of some members of
EventSenderProxy are still missing.
Comment 7 Csaba Osztrogonác 2012-09-25 05:24:06 PDT
http://trac.webkit.org/changeset/129486 unskipped a bunch of tests,
and this bug became hidden again. :-/ But I'm going to fix once I
have a day when folks don't break everything at least 5-10 times.
Comment 8 Csaba Osztrogonác 2012-11-06 22:40:08 PST

*** This bug has been marked as a duplicate of bug 101353 ***