Hi Venkat,
system statuses are defined in table TJ02 and TJ02T (text), however, these are system tables and you should not maintain entries here. There is no customer namespace for this table and the standard ABAP code will not recognize non-SAP statuses.
The link between user statuses and system statuses is via operations.
However, there are more processes which influence system statuses, not only user statuses.
For your case, I would suggest to setup Rejections. Once you Reject a line item, it will get system status Cancelled automatically. Via Rejections you can also later evaluate reasons for rejection.
Regards,
Dawood.