New post is up on the Pith Blog on the most recent test. Reposting it here because it's relevant.
v0.0.1a Post Mortem
We tested the latest version of Pith today in a non-development context. To go along with this we've tagged the latest version of the code so we have a record of its state at each of these evaluation points. After each release we'll also be writing some kind of a post mortem to go over all the technical, design, and conceptual issues that arose.
The goal over the past two months or so was to get Pith to a place where we could explore how each of its parts interacted in a discussion environment. Pith takes a familiar chat system (like Discord, Slack, iMessages, etc) and pairs it with a recursive list structure (like Notion, Workflowly, Roam Research). We arrived at a design where these two normally discrete systems are placed within the same virtual environment such that discussion takes multiple forms, from fleeting and ephemeral to preserved in as natural a way as possible. There are additional layers of deep integration between the two systems in Pith, such as transclusions which allows for any chat or document unit to reference another, and backlinks which show all units that transclude a particular unit.
We developed a system from the ground up to implement this design. While the "proper" way to go about developing a system like Pith might have been to create a series of incremental prototypes that verify parts of core functionality, we set a goal of having a minimal version of all functionality implemented and worked towards it. While our approach was risky, it seemed necessary since all the individual parts of Pith had been verified as usable, valuable components on their own. We even performed our own user testing of a prototype of the chat system in the summer and were encouraged by the results.
Returning to the present, we have mostly achieved the goal we set out at the beginning of October of implementing a minimal version of all the core functionality in the system. There's a chat, document, transclusions, and searching. It's a multi-user environment and people can contribute to each part of the system at the same time. Yet it doesn't really work.
When it comes to writing code, I am very much a hacker and will brute-force my way through to implement what needs to be implemented. I come to programming from the perspective of a designer prototyping a design; I want to get the thing to work to the point it can be played with so I can understand what it's like to use it.
The problem with Pith is that it's extremely complicated. Getting to the point of being able to experience it means designing and implementing some very complicated sub-systems that aren't visible to the user but absolutely dictate how it feels to use. @sydney and I worked on designing a system that we thought would work best given the performance, implementation, and user experience constraints that we set for ourselves or had to work with. Unfortunately, our inexperience designing these sorts of complicated systems and our very high standards resulted in a complex and fragile system. Developing the backend to a point of stability took longer that expected, as did designing robust interface between it and the frontend. This meant less time for testing and an overall less than optimal system design. As a result, we have decided the document sub-system is unsustainable to continue to develop and it will likely require a deep and complete redesign.
Pith has always been an informal project for me and Sydney. However, a series of changes have meant we've recently been looking at it with a more serious perspective. The new introduction of funding for infrastructure and testing costs from two external organizations has also played a role in pushing us to take the project in a more formal direction. As a result, we feel somewhat more obligated to do things properly, even if it means completely rewrite a large section of the system.
It wasn't all negative, though. The chat system worked much closer to what we had intended and sets a solid foundation on which plenty of other meaningful functionality can be built. The next version of Pith will include additional experimental affordances in the chat; specifically our conception of virtual non-verbal communication. Lots more to come regarding this part of the system as well.
I've certainly found Pith to be the most difficult and engaging project I've ever worked on. The deep linkage between system design and user experience is an extremely challenging space to be working in. There's many more directions we're planning on pushing the project, many of which now have even more potential with our recent external support. We'll be getting something out there soon enough.