Can we limit what an Assignee can view?

Yes.  There are several fields that can be removed, or limited to view only for the assignee.  These levels are detailed in the following article.