Why are the ActivityPub tests so complicated?
I've complained plenty on here about working on the ActivityPub tests and the difficulty it's taken for me to summon the motivation. Someone was surprised to hear that I was even talking about them be interactive... shouldn't tests just run on their own?
One reason the ActivityPub tests are so complicated is we have to test three entire suites of things:
- Test another server's "Server to server" support (requires writing a fake server and interfacing between them)
- Test another server's "client to server" support (requires operating as a client)
- Test a client... (requires operating a client->server supporting AP server) except, you can't do much to "drive" the client, you have to ask the user to do it... so, yeah, tests are interactive.
It's boring work and I'm under a deadline and I've been dreading this part of the process.
I know, complain, complain, complain. I'm doing it though.
Charles Stanhope, clacke@libranet.de ❌, Tyng-Ruey Chuang, Mike Linksvayer likes this.
clacke@libranet.de ❌, clacke@libranet.de ❌, clacke@libranet.de ❌ shared this.
@Nathan Willis I need to write a server for the client to interface with. Okay. But how do I tell your client to "now send a note" or "now favorite that" or "now delete this."
How do I do that on an android client? How do I do that on a command line client? How do I do it on a desktop client? What about a web client? I can't build hooks for all those... I don't know how your client works, and it's built to be operated by someone. We have no universal hooks mechanism for applications, as a broad stroke. And the point here is something interfacing with an API, not the other way around.
The server can still programmatically check, once the client is prompted to do something, whether or not the thing was done correctly. But I can't boop a button on someone's android app for them.
Maybe we could give someone a pile of tools and say "hey, adapt this for your application!" But would people do that? It's hard enough to convince people to run tests, let alone write them, and certainly not to wire them into an application, which is *hard work*. So having the "you do this, and I'll check if it worked right" prompting is the best option we knew of in the group.
But ... isn't this precisely what things like Selenium do?
I feel like I'm not catching whether you're saying that existing FOSS test-automation frameworks don't work, or whether there's something fundamentally different about this case. I'm not trying to be irritating; I just want to understand.
@Nathan Willis Selenium requires that a) your client be a web application and b) that all clients have the same interface, right? How would that work for an android application? How would it work if you wrote a Selenium script, and then someone wrote a web application with a completely different UI?