is writing like chocolate or like work?

I'm sensing a pattern here. I get an idea and I want to write simply for the fun of exploring this world and these characters and their emotions without any pressure for said writing to become a novel. Then I write enough about these characters that I think, wait, these people are intriguing and there is a lot going on here – maybe this can be a book. And then I try to figure out the plot, and all the random things I threw into the writing for fun suddenly need to make sense. And this has the effect of wringing much of the fun out of the writing experience.

Perhaps this pattern pre-existed my writer's block in some form, but I think it sprang from a place where I was feeling very disenchanted with writing, so writing indulgent, free-form stuff was a way to get back to my love for it. But then actually structuring that free-form love sapped my excitement. Which seems childish and not very writerly. Needless to say, I'm fretting about plot at the moment: I've got a nearly twenty-page outline ready for my next revision, and I'm struggling to muster the strength to face all the work ahead. Other ideas whisper: "write about us. have some fun. don't think about plot. it's yucky and bad."

I'm probably making plot a bigger deal than it needs to be, asking too much of my ideas too early in the game. I'm not sure. Right now I'm torn about whether I should indulge in scribbling about other ideas, or if that means I'll be cheating on my book. I don't want to get too distracted.