XCTests and Xcode 10.2


Alex Zavatone
 

I am testing upgrading our project on iOS to run in Xcode 10.2 and am seeing different behavior when running our tests.

While previously, a test target would run all the way through, login errors and successes, now errors are breaking into the debugger when errors occur and when XCTAssertTrue fails.

Reading the release notes of 10.2, I see nothing about any change in this area. Is anyone else seeing the same results?

Thanks in advance,
Alex Zavatone


Ben Kennedy
 

On 09 Apr 2019, at 10:27 am, Alex Zavatone via Groups.Io <zav@...> wrote:

While previously, a test target would run all the way through, login errors and successes, now errors are breaking into the debugger when errors occur and when XCTAssertTrue fails.

I'm still using 10.1 due to a Swift-ObjC-Swift-ObjC inheritance problem that's newly manifesting in 10.2 that I haven't yet had a chance to address, so this is only an idea, but: do you happen to have a "Test Failure" breakpoint enabled?

It turns out I have one, but I don't think I've actually ever seen it trip. Maybe that was broken before 10.2, and now it works?



Alex Zavatone
 

Breakpoints are off and there are no test failure breakpoints in the project.

Looking in the schemes, there are no arguments passed on launch but under the scheme options for Test and Diagnostics, Runtime API Checking, Main Thread Checker was enabled.

I turned it off ran the test again and it worked.  Then I turned it back on and it kept working.

OK.  That’s super odd.  Now I can’t reproduce the original issue.  

OK. So, that’s not really it.

 It turns out we had one line of code in a view controller that was required to make sure that the IBOutlets were properly wired up due to the way we are instantiating the VC.  The updater to Swift 4.2 complained about this so I commented it out.

    nativeWebBrowser?.view

If that is commented out, two of our IBOutlets that are within that view are nil and the test not only fails, but hits the debugger.

Commenting out that odd and undocumented line of code was what was causing the test to enter the debugger.

Thanks guys.


On Apr 9, 2019, at 2:08 PM, Ben Kennedy <ben-groups@...> wrote:

On 09 Apr 2019, at 10:27 am, Alex Zavatone via Groups.Io <zav@...> wrote:

While previously, a test target would run all the way through, login errors and successes, now errors are breaking into the debugger when errors occur and when XCTAssertTrue fails.

I'm still using 10.1 due to a Swift-ObjC-Swift-ObjC inheritance problem that's newly manifesting in 10.2 that I haven't yet had a chance to address, so this is only an idea, but: do you happen to have a "Test Failure" breakpoint enabled?

It turns out I have one, but I don't think I've actually ever seen it trip. Maybe that was broken before 10.2, and now it works?


<Screen Shot 2019-04-09 at 12.05.38 pm.png>


Ben Kennedy
 

On 09 Apr 2019, at 12:59 pm, Alex Zavatone via Groups.Io <zav=mac.com@groups.io> wrote:

If that is commented out, two of our IBOutlets that are within that view are nil and the test not only fails, but hits the debugger.
So the real problem was that the test was crashing due to dereferencing a nil pointer? Shouldn't that have been apparent from the stack trace?

b


Alex Zavatone
 

On Apr 9, 2019, at 3:03 PM, Ben Kennedy <ben-groups@zygoat.ca> wrote:

On 09 Apr 2019, at 12:59 pm, Alex Zavatone via Groups.Io <zav=mac.com@groups.io> wrote:

If that is commented out, two of our IBOutlets that are within that view are nil and the test not only fails, but hits the debugger.
So the real problem was that the test was crashing due to dereferencing a nil pointer? Shouldn't that have been apparent from the stack trace?

b
As I went through the process of updating the Swift syntax, all I saw was that in 10.1 we had two test failures but didn’t enter the debugger. My concern was that there was something different in 10.2, some that caused the app to enter the debugger where previously, it had succeeded. The real problem was that I split up the conversion and testing over two days and our team does not instantiate view controllers in a sane manner.

I normally code to prevent such strangeness, but our codebase is in VIPER and the programmer referenced the view that contained the IBOutlet properties to make them instantiated and didn’t document why that odd line of code (which threw a warning) was there in the first place.

People don’t normally access a view for no reason.

What slipped my mind was that the only line of code that I had commented out was that one line. I’d assumed the branch I had opened up in 10.2 was identical to the branch I cloned from and which didn’t enter the debugger when run under 10.1.