Squadly Dev Post #4, Usability Testing

Summary

Today our class conducted a usability test. In addition to classmates, testers included former students, friends, and family.

Every team was required to provide documentation that would support a test session. I created tasks and questions for testers that would break validation, demonstrate Squadly's current capabilities, and teach us about how the user interacts with it.

We were finalizing new features right up until the last hour before the class started. All was well, and then a merge from a feature branch to master broke our validation. We rushed to do what we could, and in the end our backend validations worked but our front end notifications for them were dead.

To make things worse, we didn't know the backend validations were still active until someone crashed the application by performing an invalid task. I wish we just did a git rebase to the commit before the merge was done, but instead we just ironed out all merge conflicts, tested it real quick, found out what was not working, and then just assumed that it was dead rather than spending more time testing to see what really wasn't working.

Tasks and questions that were related to testing the integrity and error handling were obviously scrapped, and while this had a huge impact on how effective this usability test could have been, we still benefited a great amount from it.

Feedback From Testing

  • The Squad Up button doesn't look like a button but as part of the cover image. What's more, the item creating from a "Squad Up" action is called an event. There is no indication near the Squad Up button that an event would be created from performing this action.

  • One tester, a friend of mine, tested it remotely. His first comment was, "what the hell does it do?" The website nor the testing document indicated what the application was. We explained Squad.ly to the tester verbally... Oops.

  • Another tester broke our application by creating a user with the same email which was part of our validation checks. Because I made error pages include content of the http request that was being made, the tester's sensitive user information was displayed for all to see... Oops again.

Detials Learned from Testing:

  • Never, ever, ever mess with feature development before a test or release without having enough time to test it our throgouhly.

  • When desigining tests with the user in mind for how the application should work, think about how to handle situations when the user experiences the app when it shouldn't work.

New Implementations:

  • A static blog of Squadly related posts was created
  • UI polish

At this point, Squadly is up and operational to the point where events and users can be made. The UI for a couple of pages was updated, and Mike was able to remove the page for the creation of new events entirely by including it in the main feed page.

For validation, Mike used the client side validations gem which can be found here.

Issues We've Faced:

Validation

  • Validation has been a nightmare to implement. Mike did a great job in updating the views and spent a lot of time adding basic validations such as duplicate emails, short passwords, etc. However, by the time the test session started, the client side aspects of the validation did not work at all. The branch for validation wasn't merged with master and we were scrambling to fix merge conflicts while our Professor was introducing the class.

Work Allotment

  • Everyone knows that validation is a big deal, and for Mike to tackle it on by himself on both the front and back end was a mistake. We had two and a half weeks to implement it. At times when Mike had other obligations to deal with, I wouldn't be able to work on it even if I wanted to since it would've taken too much time to get into. Mike was the only one who knew what was going on, and all the code for validation was his, mucking around with the code would have been bad...

    We are only a team two, and compared to other teams of three and four, our efforts are already limited. We agreed to make sure that neither of us would ever handle a core feature alone. This way, if one person was unable to work on it, the second person would be able to take over, and more importantly, both members of the group would be on the same page.