Summary report generated on 2012-01-09T14:56:25Z (UTC) by Autobahn WebSockets v0.4.10.
Pass | Test case was executed and passed successfully. |
Non-Strict | Test case was executed and passed non-strictly. A non-strict behavior is one that does not adhere to a SHOULD-behavior as described in the protocol specification or a well-defined, canonical behavior that appears to be desirable but left open in the protocol specification. An implementation with non-strict behavior is still conformant to the protocol specification. |
Fail | Test case was executed and failed. An implementation which fails a test case - other than a performance/limits related one - is non-conforming to a MUST-behavior as described in the protocol specification. |
Info | Informational test case which detects certain implementation behavior left unspecified by the spec but nevertheless potentially interesting to implementors. |
Missing | Test case is missing, either because it was skipped via the test suite configuration or deactivated, i.e. because the implementation does not implement the tested feature or breaks during running the test case. |
1 Framing | AutobahnClient/0.4.10 | wslay | ||
1.1 Text Messages | ||||
Case 1.1.1 | Pass | 1000 | Pass | 1000 |
Case 1.1.2 | Pass | 1000 | Pass | 1000 |
Case 1.1.3 | Pass | 1000 | Pass | 1000 |
Case 1.1.4 | Pass | 1000 | Pass | 1000 |
Case 1.1.5 | Pass | 1000 | Pass | 1000 |
Case 1.1.6 | Pass | 1000 | Pass | 1000 |
Case 1.1.7 | Pass | 1000 | Pass | 1000 |
Case 1.1.8 | Pass | 1000 | Pass | 1000 |
1 Framing | AutobahnClient/0.4.10 | wslay | ||
1.2 Binary Messages | ||||
Case 1.2.1 | Pass | 1000 | Pass | 1000 |
Case 1.2.2 | Pass | 1000 | Pass | 1000 |
Case 1.2.3 | Pass | 1000 | Pass | 1000 |
Case 1.2.4 | Pass | 1000 | Pass | 1000 |
Case 1.2.5 | Pass | 1000 | Pass | 1000 |
Case 1.2.6 | Pass | 1000 | Pass | 1000 |
Case 1.2.7 | Pass | 1000 | Pass | 1000 |
Case 1.2.8 | Pass | 1000 | Pass | 1000 |
2 Pings/Pongs | AutobahnClient/0.4.10 | wslay | ||
Case 2.1 | Pass | 1000 | Pass | 1000 |
Case 2.2 | Pass | 1000 | Pass | 1000 |
Case 2.3 | Pass | 1000 | Pass | 1000 |
Case 2.4 | Pass | 1000 | Pass | 1000 |
Case 2.5 | Pass | 1002 | Pass | None |
Case 2.6 | Pass | 1000 | Pass | 1000 |
Case 2.7 | Pass | 1000 | Pass | 1000 |
Case 2.8 | Pass | 1000 | Pass | 1000 |
Case 2.9 | Pass | 1000 | Pass | 1000 |
Case 2.10 | Pass | 1000 | Pass | 1000 |
Case 2.11 | Pass | 1000 | Pass | 1000 |
3 Reserved Bits | AutobahnClient/0.4.10 | wslay | ||
Case 3.1 | Pass | 1002 | Pass | 1002 |
Case 3.2 | Pass | 1002 | Non-Strict | 1002 |
Case 3.3 | Pass | 1002 | Non-Strict | 1002 |
Case 3.4 | Pass | 1002 | Non-Strict | 1002 |
Case 3.5 | Pass | 1002 | Pass | 1002 |
Case 3.6 | Pass | 1002 | Pass | 1002 |
Case 3.7 | Pass | 1002 | Pass | 1002 |
4 Opcodes | AutobahnClient/0.4.10 | wslay | ||
4.1 Non-control Opcodes | ||||
Case 4.1.1 | Pass | 1002 | Pass | 1002 |
Case 4.1.2 | Pass | 1002 | Pass | 1002 |
Case 4.1.3 | Pass | 1002 | Non-Strict | 1002 |
Case 4.1.4 | Pass | 1002 | Non-Strict | 1002 |
Case 4.1.5 | Pass | 1002 | Non-Strict | 1002 |
4 Opcodes | AutobahnClient/0.4.10 | wslay | ||
4.2 Control Opcodes | ||||
Case 4.2.1 | Pass | 1002 | Pass | 1002 |
Case 4.2.2 | Pass | 1002 | Pass | 1002 |
Case 4.2.3 | Pass | 1002 | Non-Strict | 1002 |
Case 4.2.4 | Pass | 1002 | Non-Strict | 1002 |
Case 4.2.5 | Pass | 1002 | Pass | 1002 |
5 Fragmentation | AutobahnClient/0.4.10 | wslay | ||
Case 5.1 | Pass | 1002 | Pass | None |
Case 5.2 | Pass | 1002 | Pass | None |
Case 5.3 | Pass | 1000 | Pass | 1000 |
Case 5.4 | Pass | 1000 | Pass | 1000 |
Case 5.5 | Pass | 1000 | Pass | 1000 |
Case 5.6 | Pass | 1000 | Pass | 1000 |
Case 5.7 | Pass | 1000 | Pass | 1000 |
Case 5.8 | Pass | 1000 | Pass | 1000 |
Case 5.9 | Pass | 1002 | Pass | 1002 |
Case 5.10 | Pass | 1002 | Pass | 1002 |
Case 5.11 | Pass | 1002 | Pass | 1002 |
Case 5.12 | Pass | 1002 | Pass | 1002 |
Case 5.13 | Pass | 1002 | Pass | 1002 |
Case 5.14 | Pass | 1002 | Pass | 1002 |
Case 5.15 | Pass | 1002 | Non-Strict | 1002 |
Case 5.16 | Pass | 1002 | Pass | 1002 |
Case 5.17 | Pass | 1002 | Pass | 1002 |
Case 5.18 | Pass | 1002 | Pass | 1002 |
Case 5.19 | Pass | 1000 | Pass | 1000 |
Case 5.20 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.1 Valid UTF-8 with zero payload fragments | ||||
Case 6.1.1 | Pass | 1000 | Pass | 1000 |
Case 6.1.2 | Pass | 1000 | Pass | 1000 |
Case 6.1.3 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.2 Valid UTF-8 unfragmented, fragmented on code-points and within code-points | ||||
Case 6.2.1 | Pass | 1000 | Pass | 1000 |
Case 6.2.2 | Pass | 1000 | Pass | 1000 |
Case 6.2.3 | Pass | 1000 | Pass | 1000 |
Case 6.2.4 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.3 Invalid UTF-8 differently fragmented | ||||
Case 6.3.1 | Pass | 1007 | Pass | 1007 |
Case 6.3.2 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.4 Fail-fast on invalid UTF-8 | ||||
Case 6.4.1 | Pass | 1007 | Pass | 1007 |
Case 6.4.2 | Pass | 1007 | Pass | 1007 |
Case 6.4.3 | Pass | 1007 | Pass | 1007 |
Case 6.4.4 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.5 Some valid UTF-8 sequences | ||||
Case 6.5.1 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.6 All prefixes of a valid UTF-8 string that contains multi-byte code points | ||||
Case 6.6.1 | Pass | 1007 | Pass | 1007 |
Case 6.6.2 | Pass | 1000 | Pass | 1000 |
Case 6.6.3 | Pass | 1007 | Pass | 1007 |
Case 6.6.4 | Pass | 1007 | Pass | 1007 |
Case 6.6.5 | Pass | 1000 | Pass | 1000 |
Case 6.6.6 | Pass | 1007 | Pass | 1007 |
Case 6.6.7 | Pass | 1000 | Pass | 1000 |
Case 6.6.8 | Pass | 1007 | Pass | 1007 |
Case 6.6.9 | Pass | 1000 | Pass | 1000 |
Case 6.6.10 | Pass | 1007 | Pass | 1007 |
Case 6.6.11 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.7 First possible sequence of a certain length | ||||
Case 6.7.1 | Pass | 1000 | Pass | 1000 |
Case 6.7.2 | Pass | 1000 | Pass | 1000 |
Case 6.7.3 | Pass | 1000 | Pass | 1000 |
Case 6.7.4 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.8 First possible sequence length 5/6 (invalid codepoints) | ||||
Case 6.8.1 | Pass | 1007 | Pass | 1007 |
Case 6.8.2 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.9 Last possible sequence of a certain length | ||||
Case 6.9.1 | Pass | 1000 | Pass | 1000 |
Case 6.9.2 | Pass | 1000 | Pass | 1000 |
Case 6.9.3 | Pass | 1000 | Pass | 1000 |
Case 6.9.4 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.10 Last possible sequence length 4/5/6 (invalid codepoints) | ||||
Case 6.10.1 | Pass | 1007 | Pass | 1007 |
Case 6.10.2 | Pass | 1007 | Pass | 1007 |
Case 6.10.3 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.11 Other boundary conditions | ||||
Case 6.11.1 | Pass | 1000 | Pass | 1000 |
Case 6.11.2 | Pass | 1000 | Pass | 1000 |
Case 6.11.3 | Pass | 1000 | Pass | 1000 |
Case 6.11.4 | Pass | 1000 | Pass | 1000 |
Case 6.11.5 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.12 Unexpected continuation bytes | ||||
Case 6.12.1 | Pass | 1007 | Pass | 1007 |
Case 6.12.2 | Pass | 1007 | Pass | 1007 |
Case 6.12.3 | Pass | 1007 | Pass | 1007 |
Case 6.12.4 | Pass | 1007 | Pass | 1007 |
Case 6.12.5 | Pass | 1007 | Pass | 1007 |
Case 6.12.6 | Pass | 1007 | Pass | 1007 |
Case 6.12.7 | Pass | 1007 | Pass | 1007 |
Case 6.12.8 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.13 Lonely start characters | ||||
Case 6.13.1 | Pass | 1007 | Pass | 1007 |
Case 6.13.2 | Pass | 1007 | Pass | 1007 |
Case 6.13.3 | Pass | 1007 | Pass | 1007 |
Case 6.13.4 | Pass | 1007 | Pass | 1007 |
Case 6.13.5 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.14 Sequences with last continuation byte missing | ||||
Case 6.14.1 | Pass | 1007 | Pass | 1007 |
Case 6.14.2 | Pass | 1007 | Pass | 1007 |
Case 6.14.3 | Pass | 1007 | Pass | 1007 |
Case 6.14.4 | Pass | 1007 | Pass | 1007 |
Case 6.14.5 | Pass | 1007 | Pass | 1007 |
Case 6.14.6 | Pass | 1007 | Pass | 1007 |
Case 6.14.7 | Pass | 1007 | Pass | 1007 |
Case 6.14.8 | Pass | 1007 | Pass | 1007 |
Case 6.14.9 | Pass | 1007 | Pass | 1007 |
Case 6.14.10 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.15 Concatenation of incomplete sequences | ||||
Case 6.15.1 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.16 Impossible bytes | ||||
Case 6.16.1 | Pass | 1007 | Pass | 1007 |
Case 6.16.2 | Pass | 1007 | Pass | 1007 |
Case 6.16.3 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.17 Examples of an overlong ASCII character | ||||
Case 6.17.1 | Pass | 1007 | Pass | 1007 |
Case 6.17.2 | Pass | 1007 | Pass | 1007 |
Case 6.17.3 | Pass | 1007 | Pass | 1007 |
Case 6.17.4 | Pass | 1007 | Pass | 1007 |
Case 6.17.5 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.18 Maximum overlong sequences | ||||
Case 6.18.1 | Pass | 1007 | Pass | 1007 |
Case 6.18.2 | Pass | 1007 | Pass | 1007 |
Case 6.18.3 | Pass | 1007 | Pass | 1007 |
Case 6.18.4 | Pass | 1007 | Pass | 1007 |
Case 6.18.5 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.19 Overlong representation of the NUL character | ||||
Case 6.19.1 | Pass | 1007 | Pass | 1007 |
Case 6.19.2 | Pass | 1007 | Pass | 1007 |
Case 6.19.3 | Pass | 1007 | Pass | 1007 |
Case 6.19.4 | Pass | 1007 | Pass | 1007 |
Case 6.19.5 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.20 Single UTF-16 surrogates | ||||
Case 6.20.1 | Pass | 1007 | Pass | 1007 |
Case 6.20.2 | Pass | 1007 | Pass | 1007 |
Case 6.20.3 | Pass | 1007 | Pass | 1007 |
Case 6.20.4 | Pass | 1007 | Pass | 1007 |
Case 6.20.5 | Pass | 1007 | Pass | 1007 |
Case 6.20.6 | Pass | 1007 | Pass | 1007 |
Case 6.20.7 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.21 Paired UTF-16 surrogates | ||||
Case 6.21.1 | Pass | 1007 | Pass | 1007 |
Case 6.21.2 | Pass | 1007 | Pass | 1007 |
Case 6.21.3 | Pass | 1007 | Pass | 1007 |
Case 6.21.4 | Pass | 1007 | Pass | 1007 |
Case 6.21.5 | Pass | 1007 | Pass | 1007 |
Case 6.21.6 | Pass | 1007 | Pass | 1007 |
Case 6.21.7 | Pass | 1007 | Pass | 1007 |
Case 6.21.8 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.22 Non-character code points (valid UTF-8) | ||||
Case 6.22.1 | Pass | 1000 | Pass | 1000 |
Case 6.22.2 | Pass | 1000 | Pass | 1000 |
Case 6.22.3 | Pass | 1000 | Pass | 1000 |
Case 6.22.4 | Pass | 1000 | Pass | 1000 |
Case 6.22.5 | Pass | 1000 | Pass | 1000 |
Case 6.22.6 | Pass | 1000 | Pass | 1000 |
Case 6.22.7 | Pass | 1000 | Pass | 1000 |
Case 6.22.8 | Pass | 1000 | Pass | 1000 |
Case 6.22.9 | Pass | 1000 | Pass | 1000 |
Case 6.22.10 | Pass | 1000 | Pass | 1000 |
Case 6.22.11 | Pass | 1000 | Pass | 1000 |
Case 6.22.12 | Pass | 1000 | Pass | 1000 |
Case 6.22.13 | Pass | 1000 | Pass | 1000 |
Case 6.22.14 | Pass | 1000 | Pass | 1000 |
Case 6.22.15 | Pass | 1000 | Pass | 1000 |
Case 6.22.16 | Pass | 1000 | Pass | 1000 |
Case 6.22.17 | Pass | 1000 | Pass | 1000 |
Case 6.22.18 | Pass | 1000 | Pass | 1000 |
Case 6.22.19 | Pass | 1000 | Pass | 1000 |
Case 6.22.20 | Pass | 1000 | Pass | 1000 |
Case 6.22.21 | Pass | 1000 | Pass | 1000 |
Case 6.22.22 | Pass | 1000 | Pass | 1000 |
Case 6.22.23 | Pass | 1000 | Pass | 1000 |
Case 6.22.24 | Pass | 1000 | Pass | 1000 |
Case 6.22.25 | Pass | 1000 | Pass | 1000 |
Case 6.22.26 | Pass | 1000 | Pass | 1000 |
Case 6.22.27 | Pass | 1000 | Pass | 1000 |
Case 6.22.28 | Pass | 1000 | Pass | 1000 |
Case 6.22.29 | Pass | 1000 | Pass | 1000 |
Case 6.22.30 | Pass | 1000 | Pass | 1000 |
Case 6.22.31 | Pass | 1000 | Pass | 1000 |
Case 6.22.32 | Pass | 1000 | Pass | 1000 |
Case 6.22.33 | Pass | 1000 | Pass | 1000 |
Case 6.22.34 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnClient/0.4.10 | wslay | ||
6.23 Unicode replacement character | ||||
Case 6.23.1 | Pass | 1000 | Pass | 1000 |
7 Close Handling | AutobahnClient/0.4.10 | wslay | ||
7.1 Basic close behavior (fuzzer initiated) | ||||
Case 7.1.1 | Pass | 1000 | Pass | 1000 |
Case 7.1.2 | Pass | 1000 | Pass | 1000 |
Case 7.1.3 | Pass | 1000 | Pass | 1000 |
Case 7.1.4 | Pass | 1000 | Pass | 1000 |
Case 7.1.5 | Pass | 1000 | Pass | 1000 |
Case 7.1.6 | Info | 1000 | Info | 1000 |
7 Close Handling | AutobahnClient/0.4.10 | wslay | ||
7.3 Close frame structure: payload length (fuzzer initiated) | ||||
Case 7.3.1 | Pass | 1000 | Pass | None |
Case 7.3.2 | Pass | 1002 | Pass | None |
Case 7.3.3 | Pass | 1000 | Pass | 1000 |
Case 7.3.4 | Pass | 1000 | Pass | 1000 |
Case 7.3.5 | Pass | 1000 | Pass | 1000 |
Case 7.3.6 | Pass | 1002 | Pass | None |
7 Close Handling | AutobahnClient/0.4.10 | wslay | ||
7.5 Close frame structure: payload value (fuzzer initiated) | ||||
Case 7.5.1 | Pass | 1007 | Pass | 1007 |
7 Close Handling | AutobahnClient/0.4.10 | wslay | ||
7.7 Close frame structure: valid close codes (fuzzer initiated) | ||||
Case 7.7.1 | Pass | 1000 | Pass | 1000 |
Case 7.7.2 | Pass | 1000 | Pass | 1001 |
Case 7.7.3 | Pass | 1000 | Pass | 1002 |
Case 7.7.4 | Pass | 1000 | Pass | 1003 |
Case 7.7.5 | Pass | 1000 | Pass | 1007 |
Case 7.7.6 | Pass | 1000 | Pass | 1008 |
Case 7.7.7 | Pass | 1000 | Pass | 1009 |
Case 7.7.8 | Pass | 1000 | Pass | 1010 |
Case 7.7.9 | Pass | 1000 | Pass | 1011 |
Case 7.7.10 | Pass | 1000 | Pass | 3000 |
Case 7.7.11 | Pass | 1000 | Pass | 3999 |
Case 7.7.12 | Pass | 1000 | Pass | 4000 |
Case 7.7.13 | Pass | 1000 | Pass | 4999 |
7 Close Handling | AutobahnClient/0.4.10 | wslay | ||
7.9 Close frame structure: invalid close codes (fuzzer initiated) | ||||
Case 7.9.1 | Pass | 1002 | Pass | 1002 |
Case 7.9.2 | Pass | 1002 | Pass | 1002 |
Case 7.9.3 | Pass | 1002 | Pass | 1002 |
Case 7.9.4 | Pass | 1002 | Pass | 1002 |
Case 7.9.5 | Pass | 1002 | Pass | 1002 |
Case 7.9.6 | Pass | 1002 | Pass | 1002 |
Case 7.9.7 | Pass | 1002 | Pass | 1002 |
Case 7.9.8 | Pass | 1002 | Pass | 1002 |
Case 7.9.9 | Pass | 1002 | Pass | 1002 |
Case 7.9.10 | Pass | 1002 | Pass | 1002 |
Case 7.9.11 | Pass | 1002 | Pass | 1002 |
Case 7.9.12 | Pass | 1002 | Pass | 1002 |
Case 7.9.13 | Pass | 1002 | Pass | 1002 |
7 Close Handling | AutobahnClient/0.4.10 | wslay | ||
7.13 Informational close information (fuzzer initiated) | ||||
Case 7.13.1 | Info | 1002 | Info | 1002 |
Case 7.13.2 | Info | 1002 | Info | 1002 |
9 Limits/Performance | AutobahnClient/0.4.10 | wslay | ||
9.1 Text Message (increasing size) | ||||
Case 9.1.1 | Pass 85 ms | 1000 | Pass 47 ms | 1000 |
Case 9.1.2 | Pass 330 ms | 1000 | Pass 186 ms | 1000 |
Case 9.1.3 | Pass 1312 ms | 1000 | Pass 746 ms | 1000 |
Case 9.1.4 | Pass 5252 ms | 1000 | Pass 2969 ms | 1000 |
Case 9.1.5 | Pass 10601 ms | 1000 | Pass 5869 ms | 1000 |
Case 9.1.6 | Pass 21038 ms | 1000 | Pass 11687 ms | 1000 |
9 Limits/Performance | AutobahnClient/0.4.10 | wslay | ||
9.2 Binary Message (increasing size) | ||||
Case 9.2.1 | Pass 33 ms | 1000 | Pass 22 ms | 1000 |
Case 9.2.2 | Pass 119 ms | 1000 | Pass 77 ms | 1000 |
Case 9.2.3 | Pass 469 ms | 1000 | Pass 306 ms | 1000 |
Case 9.2.4 | Pass 1884 ms | 1000 | Pass 1219 ms | 1000 |
Case 9.2.5 | Pass 3766 ms | 1000 | Pass 2433 ms | 1000 |
Case 9.2.6 | Pass 7576 ms | 1000 | Pass 4920 ms | 1000 |
9 Limits/Performance | AutobahnClient/0.4.10 | wslay | ||
9.3 Fragmented Text Message (fixed size, increasing fragment size) | ||||
Case 9.3.1 | Pass 7375 ms | 1000 | Pass 3430 ms | 1000 |
Case 9.3.2 | Pass 5815 ms | 1000 | Pass 3018 ms | 1000 |
Case 9.3.3 | Pass 5371 ms | 1000 | Pass 2915 ms | 1000 |
Case 9.3.4 | Pass 5251 ms | 1000 | Pass 2913 ms | 1000 |
Case 9.3.5 | Pass 5231 ms | 1000 | Pass 2873 ms | 1000 |
Case 9.3.6 | Pass 5251 ms | 1000 | Pass 2878 ms | 1000 |
Case 9.3.7 | Pass 5218 ms | 1000 | Pass 2894 ms | 1000 |
Case 9.3.8 | Pass 5216 ms | 1000 | Pass 2895 ms | 1000 |
Case 9.3.9 | Pass 5227 ms | 1000 | Pass 2906 ms | 1000 |
9 Limits/Performance | AutobahnClient/0.4.10 | wslay | ||
9.4 Fragmented Binary Message (fixed size, increasing fragment size) | ||||
Case 9.4.1 | Pass 3875 ms | 1000 | Pass 1722 ms | 1000 |
Case 9.4.2 | Pass 2406 ms | 1000 | Pass 1337 ms | 1000 |
Case 9.4.3 | Pass 1956 ms | 1000 | Pass 1209 ms | 1000 |
Case 9.4.4 | Pass 1863 ms | 1000 | Pass 1193 ms | 1000 |
Case 9.4.5 | Pass 1833 ms | 1000 | Pass 1196 ms | 1000 |
Case 9.4.6 | Pass 1831 ms | 1000 | Pass 1172 ms | 1000 |
Case 9.4.7 | Pass 1833 ms | 1000 | Pass 1163 ms | 1000 |
Case 9.4.8 | Pass 1830 ms | 1000 | Pass 1198 ms | 1000 |
Case 9.4.9 | Pass 1824 ms | 1000 | Pass 1189 ms | 1000 |
9 Limits/Performance | AutobahnClient/0.4.10 | wslay | ||
9.5 Text Message (fixed size, increasing chop size) | ||||
Case 9.5.1 | Pass 1863 ms | 1000 | Pass 1698 ms | 1000 |
Case 9.5.2 | Pass 1355 ms | 1000 | Pass 1210 ms | 1000 |
Case 9.5.3 | Pass 1343 ms | 1000 | Pass 965 ms | 1000 |
Case 9.5.4 | Pass 1346 ms | 1000 | Pass 847 ms | 1000 |
Case 9.5.5 | Pass 1317 ms | 1000 | Pass 789 ms | 1000 |
Case 9.5.6 | Pass 1330 ms | 1000 | Pass 764 ms | 1000 |
9 Limits/Performance | AutobahnClient/0.4.10 | wslay | ||
9.6 Binary Text Message (fixed size, increasing chop size) | ||||
Case 9.6.1 | Pass 1457 ms | 1000 | Pass 1292 ms | 1000 |
Case 9.6.2 | Pass 965 ms | 1000 | Pass 832 ms | 1000 |
Case 9.6.3 | Pass 733 ms | 1000 | Pass 551 ms | 1000 |
Case 9.6.4 | Pass 598 ms | 1000 | Pass 426 ms | 1000 |
Case 9.6.5 | Pass 536 ms | 1000 | Pass 367 ms | 1000 |
Case 9.6.6 | Pass 507 ms | 1000 | Pass 335 ms | 1000 |
9 Limits/Performance | AutobahnClient/0.4.10 | wslay | ||
9.7 Text Message Roundtrip Time (fixed number, increasing size) | ||||
Case 9.7.1 | Pass 248 ms | 1000 | Pass 114 ms | 1000 |
Case 9.7.2 | Pass 320 ms | 1000 | Pass 139 ms | 1000 |
Case 9.7.3 | Pass 417 ms | 1000 | Pass 181 ms | 1000 |
Case 9.7.4 | Pass 800 ms | 1000 | Pass 317 ms | 1000 |
Case 9.7.5 | Pass 2296 ms | 1000 | Pass 870 ms | 1000 |
Case 9.7.6 | Pass 8226 ms | 1000 | Pass 3129 ms | 1000 |
9 Limits/Performance | AutobahnClient/0.4.10 | wslay | ||
9.8 Binary Message Roundtrip Time (fixed number, increasing size) | ||||
Case 9.8.1 | Pass 260 ms | 1000 | Pass 113 ms | 1000 |
Case 9.8.2 | Pass 293 ms | 1000 | Pass 130 ms | 1000 |
Case 9.8.3 | Pass 333 ms | 1000 | Pass 148 ms | 1000 |
Case 9.8.4 | Pass 486 ms | 1000 | Pass 212 ms | 1000 |
Case 9.8.5 | Pass 1047 ms | 1000 | Pass 456 ms | 1000 |
Case 9.8.6 | Pass 3233 ms | 1000 | Pass 1425 ms | 1000 |
10 Autobahn Protocol Options | AutobahnClient/0.4.10 | wslay | ||
10.1 Auto-Fragmentation | ||||
Case 10.1.1 | Pass | 1000 | Pass | 1000 |
Case Description
Send text message with payload 0.
Case Expectation
Receive echo'ed text message (with empty payload). Clean close with normal code.
Case Description
Send text message message with payload of length 125.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send text message message with payload of length 126.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send text message message with payload of length 127.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send text message message with payload of length 128.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send text message message with payload of length 65535.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send text message message with payload of length 65536.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send text message message with payload of length 65536. Sent out data in chops of 997 octets.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send binary message with payload 0.
Case Expectation
Receive echo'ed binary message (with empty payload). Clean close with normal code.
Case Description
Send binary message message with payload of length 125.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send binary message message with payload of length 126.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send binary message message with payload of length 127.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send binary message message with payload of length 128.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send binary message message with payload of length 65535.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send binary message message with payload of length 65536.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send binary message message with payload of length 65536. Sent out data in chops of 997 octets.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send ping without payload.
Case Expectation
Pong (with empty payload) is sent in reply to Ping. Clean close with normal code.
Case Description
Send ping with small text payload.
Case Expectation
Pong with payload echo'ed is sent in reply to Ping. Clean close with normal code.
Case Description
Send ping with small binary (non UTF-8) payload.
Case Expectation
Pong with payload echo'ed is sent in reply to Ping. Clean close with normal code.
Case Description
Send ping with binary payload of 125 octets.
Case Expectation
Pong with payload echo'ed is sent in reply to Ping. Clean close with normal code.
Case Description
Send ping with binary payload of 126 octets.
Case Expectation
Connection is failed immediately (1002/Protocol Error), since control frames are only allowed to have payload up to and including 125 octets..
Case Description
Send ping with binary payload of 125 octets, send in octet-wise chops.
Case Expectation
Pong with payload echo'ed is sent in reply to Ping. Implementations must be TCP clean. Clean close with normal code.
Case Description
Send unsolicited pong without payload. Verify nothing is received. Clean close with normal code.
Case Expectation
Nothing.
Case Description
Send unsolicited pong with payload. Verify nothing is received. Clean close with normal code.
Case Expectation
Nothing.
Case Description
Send unsolicited pong with payload. Send ping with payload. Verify pong for ping is received.
Case Expectation
Nothing in reply to own Pong, but Pong with payload echo'ed in reply to Ping. Clean close with normal code.
Case Description
Send 10 Pings with payload.
Case Expectation
Pongs for our Pings with all the payloads. Note: This is not required by the Spec .. but we check for this behaviour anyway. Clean close with normal code.
Case Description
Send 10 Pings with payload. Send out octets in octet-wise chops.
Case Expectation
Pongs for our Pings with all the payloads. Note: This is not required by the Spec .. but we check for this behaviour anyway. Clean close with normal code.
Case Description
Send small text message with RSV = 1.
Case Expectation
The connection is failed immediately (1002/protocol error), since RSV must be 0, when no extension defining RSV meaning has been negoiated.
Case Description
Send small text message, then send again with RSV = 2, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since RSV must be 0, when no extension defining RSV meaning has been negoiated. The Pong is not received.
Case Description
Send small text message, then send again with RSV = 3, then send Ping. Octets are sent in frame-wise chops. Octets are sent in octet-wise chops.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since RSV must be 0, when no extension defining RSV meaning has been negoiated. The Pong is not received.
Case Description
Send small text message, then send again with RSV = 4, then send Ping. Octets are sent in octet-wise chops.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since RSV must be 0, when no extension defining RSV meaning has been negoiated. The Pong is not received.
Case Description
Send small binary message with RSV = 5.
Case Expectation
The connection is failed immediately, since RSV must be 0.
Case Description
Send Ping with RSV = 6.
Case Expectation
The connection is failed immediately, since RSV must be 0.
Case Description
Send Close with RSV = 7.
Case Expectation
The connection is failed immediately, since RSV must be 0.
Case Description
Send frame with reserved non-control Opcode = 3.
Case Expectation
The connection is failed immediately.
Case Description
Send frame with reserved non-control Opcode = 4 and non-empty payload.
Case Expectation
The connection is failed immediately.
Case Description
Send small text message, then send frame with reserved non-control Opcode = 5, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since reserved opcode frame is used. A Pong is not received.
Case Description
Send small text message, then send frame with reserved non-control Opcode = 6 and non-empty payload, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since reserved opcode frame is used. A Pong is not received.
Case Description
Send small text message, then send frame with reserved non-control Opcode = 7 and non-empty payload, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since reserved opcode frame is used. A Pong is not received.
Case Description
Send frame with reserved control Opcode = 11.
Case Expectation
The connection is failed immediately.
Case Description
Send frame with reserved control Opcode = 12 and non-empty payload.
Case Expectation
The connection is failed immediately.
Case Description
Send small text message, then send frame with reserved control Opcode = 13, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since reserved opcode frame is used. A Pong is not received.
Case Description
Send small text message, then send frame with reserved control Opcode = 14 and non-empty payload, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since reserved opcode frame is used. A Pong is not received.
Case Description
Send small text message, then send frame with reserved control Opcode = 15 and non-empty payload, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since reserved opcode frame is used. A Pong is not received.
Case Description
Send Ping fragmented into 2 fragments.
Case Expectation
Connection is failed immediately, since control message MUST NOT be fragmented.
Case Description
Send Pong fragmented into 2 fragments.
Case Expectation
Connection is failed immediately, since control message MUST NOT be fragmented.
Case Description
Send text Message fragmented into 2 fragments.
Case Expectation
Message is processed and echo'ed back to us.
Case Description
Send text Message fragmented into 2 fragments, octets are sent in frame-wise chops.
Case Expectation
Message is processed and echo'ed back to us.
Case Description
Send text Message fragmented into 2 fragments, octets are sent in octet-wise chops.
Case Expectation
Message is processed and echo'ed back to us.
Case Description
Send text Message fragmented into 2 fragments, one ping with payload in-between.
Case Expectation
A pong is received, then the message is echo'ed back to us.
Case Description
Send text Message fragmented into 2 fragments, one ping with payload in-between. Octets are sent in frame-wise chops.
Case Expectation
A pong is received, then the message is echo'ed back to us.
Case Description
Send text Message fragmented into 2 fragments, one ping with payload in-between. Octets are sent in octet-wise chops.
Case Expectation
A pong is received, then the message is echo'ed back to us.
Case Description
Send unfragmented Text Message after Continuation Frame with FIN = true, where there is nothing to continue, sent in one chop.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send unfragmented Text Message after Continuation Frame with FIN = true, where there is nothing to continue, sent in per-frame chops.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send unfragmented Text Message after Continuation Frame with FIN = true, where there is nothing to continue, sent in octet-wise chops.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send unfragmented Text Message after Continuation Frame with FIN = false, where there is nothing to continue, sent in one chop.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send unfragmented Text Message after Continuation Frame with FIN = false, where there is nothing to continue, sent in per-frame chops.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send unfragmented Text Message after Continuation Frame with FIN = false, where there is nothing to continue, sent in octet-wise chops.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send text Message fragmented into 2 fragments, then Continuation Frame with FIN = false where there is nothing to continue, then unfragmented Text Message, all sent in one chop.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Repeated 2x: Continuation Frame with FIN = false (where there is nothing to continue), then text Message fragmented into 2 fragments.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Repeated 2x: Continuation Frame with FIN = true (where there is nothing to continue), then text Message fragmented into 2 fragments.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send text Message fragmented into 2 fragments, with both frame opcodes set to text, sent in one chop.
Case Expectation
The connection is failed immediately, since all data frames after the initial data frame must have opcode 0.
Case Description
A fragmented text message is sent in multiple frames. After
sending the first 2 frames of the text message, a Ping is sent. Then we wait 1s,
then we send 2 more text fragments, another Ping and then the final text fragment.
Everything is legal.
Case Expectation
The peer immediately answers the first Ping before
it has received the last text message fragment. The peer pong's back the Ping's
payload exactly, and echo's the payload of the fragmented message back to us.
Case Description
Same as Case 5.19, but send all frames with SYNC = True.
Note, this does not change the octets sent in any way, only how the stream
is chopped up on the wire.
Case Expectation
Same as Case 5.19. Implementations must be agnostic to how
octet stream is chopped up on wire (must be TCP clean).
Case Description
Send text message of length 0.
Case Expectation
A message is echo'ed back to us (with empty payload).
Case Description
Send fragmented text message, 3 fragments each of length 0.
Case Expectation
A message is echo'ed back to us (with empty payload).
Case Description
Send fragmented text message, 3 fragments, first and last of length 0, middle non-empty.
Case Expectation
A message is echo'ed back to us (with payload = payload of middle fragment).
Case Description
Send a valid UTF-8 text message in one fragment.
MESSAGE:
Hello-µ@ßöäüà á-UTF-8!!
48656c6c6f2dc2b540c39fc3b6c3a4c3bcc3a0c3a12d5554462d382121
Case Expectation
The message is echo'ed back to us.
Case Description
Send a valid UTF-8 text message in two fragments, fragmented on UTF-8 code point boundary.
MESSAGE FRAGMENT 1:
Hello-µ@ßöä
48656c6c6f2dc2b540c39fc3b6c3a4
MESSAGE FRAGMENT 2:
üà á-UTF-8!!
c3bcc3a0c3a12d5554462d382121
Case Expectation
The message is echo'ed back to us.
Case Description
Send a valid UTF-8 text message in fragments of 1 octet, resulting in frames ending on positions which are not code point ends.
MESSAGE:
Hello-µ@ßöäüà á-UTF-8!!
48656c6c6f2dc2b540c39fc3b6c3a4c3bcc3a0c3a12d5554462d382121
Case Expectation
The message is echo'ed back to us.
Case Description
Send a valid UTF-8 text message in fragments of 1 octet, resulting in frames ending on positions which are not code point ends.
MESSAGE:
κόσμε
cebae1bdb9cf83cebcceb5
Case Expectation
The message is echo'ed back to us.
Case Description
Send invalid UTF-8 text message unfragmented.
MESSAGE:
Îºá½¹ÏƒÎ¼Îµí €edited
cebae1bdb9cf83cebcceb5eda080656469746564
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send invalid UTF-8 text message in fragments of 1 octet, resulting in frames ending on positions which are not code point ends.
MESSAGE:
Îºá½¹ÏƒÎ¼Îµí €edited
cebae1bdb9cf83cebcceb5eda080656469746564
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send invalid UTF-8 text message in 3 fragments (frames).
First frame payload is valid, then wait, then 2nd frame which contains the payload making the sequence invalid, then wait, then 3rd frame with rest.
Note that PART1 and PART3 are valid UTF-8 in themselves, PART2 is a 0x11000 encoded as in the UTF-8 integer encoding scheme, but the codepoint is invalid (out of range).
MESSAGE PARTS:
PART1 = κόσμε (cebae1bdb9cf83cebcceb5)
PART2 = ô€€ (f4908080)
PART3 = edited (656469746564)
Case Expectation
The first frame is accepted, we expect to timeout on the first wait. The 2nd frame should be rejected immediately (fail fast on UTF-8). If we timeout, we expect the connection is failed at least then, since the complete message payload is not valid UTF-8.
Case Description
Same as Case 6.4.1, but in 2nd frame, we send only up to and including the octet making the complete payload invalid.
MESSAGE PARTS:
PART1 = κόσμεô (cebae1bdb9cf83cebcceb5f4)
PART2 = (90)
PART3 = €€edited (8080656469746564)
Case Expectation
The first frame is accepted, we expect to timeout on the first wait. The 2nd frame should be rejected immediately (fail fast on UTF-8). If we timeout, we expect the connection is failed at least then, since the complete message payload is not valid UTF-8.
Case Description
Same as Case 6.4.1, but we send message not in 3 frames, but in 3 chops of the same message frame.
MESSAGE PARTS:
PART1 = κόσμε (cebae1bdb9cf83cebcceb5)
PART2 = ô€€ (f4908080)
PART3 = edited (656469746564)
Case Expectation
The first chop is accepted, we expect to timeout on the first wait. The 2nd chop should be rejected immediately (fail fast on UTF-8). If we timeout, we expect the connection is failed at least then, since the complete message payload is not valid UTF-8.
Case Description
Same as Case 6.4.2, but we send message not in 3 frames, but in 3 chops of the same message frame.
MESSAGE PARTS:
PART1 = κόσμεô (cebae1bdb9cf83cebcceb5f4)
PART2 = (90)
PART3 = ()
Case Expectation
The first chop is accepted, we expect to timeout on the first wait. The 2nd chop should be rejected immediately (fail fast on UTF-8). If we timeout, we expect the connection is failed at least then, since the complete message payload is not valid UTF-8.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE:
κόσμε
cebae1bdb9cf83cebcceb5
Case Expectation
The message is echo'ed back to us.
Case Description
Send a text message with payload which is not valid UTF-8 in one fragment.
MESSAGE:
Î
ce
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE:
κ
ceba
Case Expectation
The message is echo'ed back to us.
Case Description
Send a text message with payload which is not valid UTF-8 in one fragment.
MESSAGE:
뼇
cebae1
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send a text message with payload which is not valid UTF-8 in one fragment.
MESSAGE:
κá½
cebae1bd
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE:
κό
cebae1bdb9
Case Expectation
The message is echo'ed back to us.
Case Description
Send a text message with payload which is not valid UTF-8 in one fragment.
MESSAGE:
κόÏ
cebae1bdb9cf
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE:
κόσ
cebae1bdb9cf83
Case Expectation
The message is echo'ed back to us.
Case Description
Send a text message with payload which is not valid UTF-8 in one fragment.
MESSAGE:
κόσÎ
cebae1bdb9cf83ce
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE:
κόσμ
cebae1bdb9cf83cebc
Case Expectation
The message is echo'ed back to us.
Case Description
Send a text message with payload which is not valid UTF-8 in one fragment.
MESSAGE:
κόσμÎ
cebae1bdb9cf83cebcce
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE:
κόσμε
cebae1bdb9cf83cebcceb5
Case Expectation
The message is echo'ed back to us.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE: