The way you described the current responsibility rule functionality is correct. You would need to create two separate responsibility rules. One rule to provide general read access to assets when Confidential = No, and another rule to provide read access to assets when Confidential = Yes.
Since when filters currently only support the equals operator, your best option for the Confidential flag would be a List type, with Yes/No values, and with No set as the default value.
Having a more streamlined way to manage this would be nice to have.