SOLVED. Historical novel focused on a hat seller's family [s]

[Adi]

I read this book a really long time ago (6-8 years ago).It was a really old looking book with no cover. But it had the starting and what I assumed,at that time, was the ending pages of the book, which later turned out to be the ending page of some chapter.I really have no idea how many pages were missing.
My memory regarding the story is a bit spotty but I'll try to note down whatever I can (hopefully in sequence).

-Setting: Britain (I'm assuming)
-The Antagonist: The hat seller.Strict, grumpy man.Class and status is important to him. His family members include his mother, wife, son(who gets sent to India),his daughter Mary (who falls in love with some local guy) and a youngest daughter who is very good in studies.
- The protagonist: Mary, the daughter(I'm pretty sure this is her name, but I could be wrong)
- Plot(the parts I can remember)
(view spoiler)[
* The hatter is shown as an evil guy who thinks his son is useless and doesn't really like his wife either. He only likes his youngest daughter and wants her to win some scholarship, which has never been awarded to a girl, so he can show off to this other guy whose son is also attempting to get it.He has a young mistress(a prostitute?) he visits regularly.
*Hatter's son goes to India(as a soldier/captain?). I think it was to escape his father.His mother adores him and he keeps asking her money from India for gambling and other such things. His mother ends up taking a large loan from a bank with no knowledge about the gambling. She doesn't tell the hatter about the loan.
* Mary gets pregnant and tries to hide it from her family. The guy she likes goes out of town for some job.Her father finds out about it after few months and kicks her out of the house during a storm.She wanders around in the storm and ends up in some cattle shed. A doctor is called but the baby dies.Around the same time her lover dies in an train accident.
*The hatter's son return after lot of gambling and spending time in various brothel.His mother realizes that he has wasted all the money and then falls sick.The hatter finds out and blames her.She eventually dies. The mistress ends up cooking for the hatter, which she doesn't like. She ends up eloping with the hatter's son.
*Mary returns(after some time?) and meets with the doctor who saved her life. She goes back to her house because of her sister and grandmother.Then a romance between Mary and the doctor is hinted at.
*The hatter loses his business to a new and modern company.He ends up throwing all his remaining hats at people walking on the street. He puts all his hope(for revival of prestige?) on his youngest daughter and the scholarship. In the end, she doesn't get the scholarship and tries to hang herself.Mary finds her and sends for the doctor.
(hide spoiler)]

This is how far I got in the book.Does anyone know this book?
*fingers crossed*

[Ayshe]

I think I have read only the second part but this sounds like the Hatter's Castle

[Adi]

Ayshe wrote: "I think I have read only the second part but this sounds like the Hatter's Castle"

It is!!
Thank you so much.If I could, I would hug you right now!This story has been nagging me for so long.