We all must be ruthless with our time. For it is something that can be leveraged and purchased, yet never increased incrementally. You only have so much - no matter how efficiently it is used, it is used nonetheless.
We have a limited time on this planet, you and I. How we choose to use it, be it writing in an effort to help, or reading in an effort to grow, we are all using time at this very moment.
It is perfectly fine, and great to be generous with your time, so long as it is aligned with your life's purpose. Mine, involves teaching, which means that I can be generous with my writing and other methods I employ of passing on knowledge while inspiring creative action.
I do not feel guilty in saying "no" to someone when I perceive their request to be outside of my life's purpose, and neither should you. Their request is within someone else's life's purpose. They'll do it better than us, anyway, as they're driven by their own passions.
Do not perceive someone saying "no" to you as selfish or arrogant, it is merely a prioritizing. Even Superman can't save everyone, so, he chooses to primarily save those he's best suited to save. His powers are better used stopping asteroids from hitting the Earth, than breaking up a mugging. When there's a mystery to be solved or strategy to be created, he delegates that to Batman. Batman, in turn, delegates the heavy lifting to Superman, and the world is better off.
Ask yourself: Is there an activity in my life I am doing right now that isn't giving me the results or satisfaction I thought it would? Should I discontinue it? Saying no is a skill, which is often underdeveloped.
Recommended Reading: The 4 Hour Work Week
Sensei Ono, Shinka Martial Arts
How could you best encourage a blogger to blog?