I think I have the same mindset with you in this regard, whatever I am not doing any work I always feel guilty. That is why I always try as much as possible to do work because it makes me feel some level of progress.If I have taken an intentional break from work, I just feel relaxed and happy. But if I am not doing work due to procrastination and laziness, I feel very guilty. It can even affect my mental health.