Hi Justin,
I understand what you're saying. We're in the process doing some design work on workflow, including a better way to work assignments. While any changes may be out a few months in the meantime a potential alternative is to work assignments from the Workflow Items screen. You'd just have to filter on yourself as the assignee. That screen has every detail of every step in the workflow including emails, form responses, any field changes etc..
Let me know if this helps.
Thanks,
Mandy Bank, Product Management