Word frequency tool

[Ann, Supreme Overlord]

This looks great! I'm definitely going to use this for Hollow Towns :)

[Bryan]

I intend to give your word use counter a try. I currently rely on my wife who proof reads for me and I'm sure she will be very grateful if I can iron out one of my many faults which she corrects for me.

Best regards

Bryan

[Bryan]

Just tried your machine. What a great idea! I'm sure this will save my poor wife a deal of underlining.

Thanks

[J.D.]

the response here is really gratifying...I find it really useful but if it helps other people it just reinforces the worth in spending the time to come up with it. Given the level of response here I'll start spending a little time improving it...

thanks all

[Owen]

J.D. wrote: "the response here is really gratifying...I find it really useful but if it helps other people it just reinforces the worth in spending the time to come up with it. Given the level of response here ..."

Keep us posted. As a non-chrome user, I'm interested. Currently, we do manual searches in Word for the "word of the day" (I like "vast" way too much at times) but I'd be interested to see what your tool kicks out. (And I'll admit I'm a bit curious about your algorithm. I can think of ways to do this using MySQL, but I don't know what tools JavaScript provides -- they obviously work!)

[Bryan]

Having used the word frequency tool a couple of times , have you considered asking publishers of the various thesauruses for a royalty for boosting their sales. Just a thought.

[Winter]

I'm adding my voice to the chorus, JD. This is a great tool! I have this intense love affair with "just" and this might just lead to a break up.

(see? it keeps sneaking in there)

:)

[J.D.]

So I'm in the process of putting together a new version that gives more details on the "hotspots". If you leave your cursor over the hotspot number in the third column, it will raise a tool tip that will list the word count positions of the occurrences of the word. I'm not pleased with this UI presentation so I'm going to square that away before putting it up.

@Owen I have been able to reproduce the issue you have seen with Firefox. It's mysterious so I'm having to debug further on it, but I did find that it was failing for me on the word "watch" and if I went through my text and changed the letters 'watch' to anything else it runs fine. It's tricky to debug but I'm on it.

@Bryan that's a nice idea. Maybe I could add a feature where if you click the word it opens the thesaurus in another window with the word already entered. Then I could get royalties from the thesaurus website and retire early. Well, ok, it's a nice dream but unlikely :) still an interesting feature...

[J.D.]

New version is up. It has the not so entirely pretty interface to show word count locations of hotspot words and more importantly it fixes a firefox bug.

@Owen, Let me know if this fixes what you were seeing. It should. If not let me know and we can chat privately to figure it out.

thanks!

[J.D.]

Ok, one final thing and then I'll leave you guys alone to play with it. I turned the words in the list of results into links that will take you to Thesaurus.com with the word already in the view (including results of that)

[Dwayne, Head of Lettuce]

J.D. wrote: "Ok, one final thing and then I'll leave you guys alone to play with it. I turned the words in the list of results into links that will take you to Thesaurus.com with the word already in the view (i..."

Oh, cool idea, J.D.!

[Winter]

That's a great mod! Thanks, JD!

[Riley, Viking Extraordinaire]

This tells you how frequently you use words S.M. so over used words can be changed.

[Jenycka]

That thing is freaking awesome! Thanks!

[Christina]

S.M. wrote: "Yes, I got that ... "fInd" will tell you how many times a word appears too ... Just saying ..."

If you know which word you are looking for, then yes, you could use the find feature in Word. However, the tool that JD has built and is sharing freely with us, shows you which words you are overusing and at what frequency.

[Dwayne, Head of Lettuce]

S.M. wrote: "Yes, I got that ... "fInd" will tell you how many times a word appears too ... Just saying ..."

Well, you'd have to enter every word in your story one at a time into Word to get the results of J.D.'s marvelous contraption. Try it sometime!

[Igzy]

I don't have anything constructive to add beyond 'this is so cool, thank you for sharing it!'

Wonderfully useful.

[Sherri]

what Igzy said, super cool! thanks for sharing.

[Jenycka]

I put the draft of my latest erotic novel through there... wow there are some words I over use, but none of them ought to be shared here....

[Winter]

Jenyca, that's friggin' hilarious.

[J.D.]

Yeah, I agree, that's hilarious :)

[Christina]

I just finished my first draft of my next book and I'm almost afraid to see what this says. :)

[J.D.]

congrats on your first draft--magical moment :) And take the output of this with a grain of salt. It's just munging numbers, etc, with no semantic analysis.

[C.B., Beach Body Moderator]

Jenycka wrote: "I put the draft of my latest erotic novel through there... wow there are some words I over use, but none of them ought to be shared here...."

I apparently have a very vivid imagination when it comes to... well...
words and stuff...

;)

[Christina]

Is there a way to check the frequency of combinations of words? I'm noticing I really like using certain phrases too much. Currently it's 'hairy eyeball'.

[C.B., Beach Body Moderator]

Christina wrote: "Is there a way to check the frequency of combinations of words? I'm noticing I really like using certain phrases too much. Currently it's 'hairy eyeball'."

I use 'Thick Ropes' too much... in... my... nautical... adventure.

[J.D.]

Christina,

I will have to think on that idea. I like it a lot--I do the same thing. I just have to figure out how to implement it :)

[J.D.]

So one final update (for a while--gotta get back to my manuscript).

I changed drastically how you choose the built in exclusion list. Previously it was just a checkbox and you got all or none of the list. And there was no way to know what words were on that list. I've altered things so that the exclusion list is broken up into "sections". There are five checkboxes and you can select none or all of them. For example, the first one is 'exclude articles." if you mouse over the orange icon next to the checkbox text it will show a tooltip listing what words will be ignored.

Also, the tooltips are much more user friendly now in terms of appearance.

Thanks for all the input--it's always easier to improve software when you have REAL user feedback. And keep the ideas coming. I've got to put this aside for a while to focus on my next book, but I'll always come back and improve this tool when I have some free time :)

[Riley, Viking Extraordinaire]

Awesome JD, I will be adding this to the author resource at some point.

[Ann, Supreme Overlord]

I'm excited to try this out!

[J.D.]

cool :) Let me know what you think--any issues you find that need to be fixed, etc.

[L.J.]

I started developing something in 2003 that relied on the GNU concordance/ptx package and then analysed the results from their output to auto-determine frequencies of both words and phrases, and then report on any that were used too close together in the text (i.e. not just the number of uses, but how frequently through the text). I was still playing around with the tuning - since it's okay to use an unusual word if it's infrequent, but reusing a truly unique phrase even once is the wrong thing to do. It's unfinished, and I doubt if you would be interested, but if you might be, I'd be happy to send them along if you want to look at them. The heavy lifting is done by a 500-line ICI script, which you're unlikely to be familiar with, but is similar to Javascript.

[P.L.]

Sounds interesting and helpful. I'm going to give it a shot later.

[Bret]

Handy tool, thanks! I have a nasty tendency to repeat phrases and use those stodgy old thinking words like 'realised' too much.

[J.D.]

I'll definitely be checking this out. Word repetition is actually something I'm starting to look for when editing, and this can only help.

[Justin]

I just did it and wow..."the" and "I" come up with the biggest amounts..I'll have to filter them out next time.

Very neat tool though it can certainly be helpful.

[K.P.]

How interesting... I ran it on my first book I published and just looking at the hotspots were eye-opening. Comparing it to the sequels, my writing's improved :D

Cool tool, bro.

[Judy]

I've used word counting tools before, but this is by far the best. I like that I can exclude certain words. Thanks for sharing this amazing tool.

[Leon]

Great tool, thanks.
I am soooo thankful to find out that I'm not the only one guilty of word frequency.

[S.D.]

That is a very useful tool. I must say reading through the post had me in fits of giggles at times.

[Grace]

This saved me from a lot of red ink on several assignments and my master's thesis. Now, I'm going to use it for a bigger project: a book. I can't tell you how awesome this tool has been for me.

[Diane]

This tool is awesome! I overused the words "I" and "my" but since I am writing in first person, it might be alright. Regardless, I may reduce the number. Thank you J.D.

[J.D.]

Wow! The response to this has been so positive! I'm really pleased that it's been useful and helpful to people. I've got so many things I'd like to add to it but I'm honestly worried about overloading the browser process--this all runs on your browser in javascript.

I'm also in the far advance planning stages of coming up with a more comprehensive MacOs tool...that way I could pack more features into it and make it more powerful...

[Mark]

Great tool. I discovered that my characters were always going down or crossing the street, but never once did they use the sidewalk. Ouch.

[Bacil]

J.D. wrote: "Wow! The response to this has been so positive! I'm really pleased that it's been useful and helpful to people. I've got so many things I'd like to add to it but I'm honestly worried about overload..."

I was just going to offer that when you do put it up on github or bitbucket, holler. I was thinking about translating it to a Swift Mac OS X/iOS app, and since it's under the MIT just roll that code back into github, as well.

[Brittney]

Going to have to bookmark this for sure. I too tend to find a word that I like as a description and then find myself wondering, as I write, if I have used it too much. I am eager to be able to add this to my writing tools as I work on my next novel!

[Anika]

i write everything in iwriter. keeps track of many things included number of times you use a word.

[Doug]

This is pretty dang cool.

[Jacob]

Sounds like a really useful tool. I often notice myself using several words all the time and sometimes close together so it's nice to find something to help with that. Thanks for sharing!

[Benita]

Thank you so much for sharing this, J.D.! No question on my part about bookmarking it. :) I definitely have a problem with overusing some words so this will be super helpful when I get to the editing stage of my WIP.