Ok, this is the one post that I will take the time to respond to because I personally went through this, and I represented myself (I was my own lawyer) in my court battles through Family Court, and Superior Court. I am going to give you facts, so do not confuse them with 'uncaring'; I have been in & out of court with my ex many times since January 2006 because he always has something new to complain about every 2 years.
1) No matter what a POS your ex-BF is, he has equal rights to his child
2) Unless there is a police record or a conviction involving drug use, the court will not force supervised access unless there is factual information to state otherwise; your statement, or family/friends vouching for what you're saying doesn't change anything
3) Family court Judges have heard everything, there is nothing new that you could possibly say that they have not already heard in another case. Unfortunately in many custody cases, things get very ugly and it stops being about the children, and more about how to get back at the other spouse
4) No matter what you now say about how he treated your daughter, yourself, the child you have together, the fact remains that your actions of remaining in the relationship is a silent agreement that you did not feel that the children were in danger being around him; this gives more reason for the court to not issue supervised access.
5) What is very important in custody cases is something called 'Status Quo'; typically, the courts do not like to make too many changes to this as it can negatively affect the children.
6) What you presently have right now is called 'Defacto Custody'; what you need to be aware of right now is that your ex-bf has the same rights to the child you had together as you do. If he lives somewhere right now where he can set up a bed for that child in a safe environment (roof over her head, food, clothes etc - remember, safe in the eyes of the law is specific) then he could legally pick up that child for a 'visit' and refuse to return her. The police typically will not get too involved in those type of matters, they can only suggest that he return the child, but they cannot force him to do as such.
7) Knowing that information above, it's also not a good idea to refuse him access to his child; you can certainly set up something in a public location and bring friends/family with you for protection if need be. If he has made physical threats though, then stay away from him for now.
8) Starting right now, get yourself a journal; you need to document everything in point form. All conversations, all texts, phone calls, absolutely everything; this can be your saving grace when you/your lawyer needs to file the affadivits with the courts, having detailed information makes a huge difference in the outcome of most cases.
9) Do not post negative/derogatory messages about him on Facebook etc, this can work against you in court as it can be used as evidence as well.
10) His claim on your home will depend on how long you have lived together. If you have hit the 2 year mark (under BC law) then the division of property falls under what is called the law of trust. It means he may have a right to a part of the property if he contributed to it. Even if it's been less than 2 years, he might be able to file under 'unjust enrichment', but those are not always successful with short term relationships.
It's late so there is much more I could add, but it would take quite awhile.
Just as an FYI - I'm fully supporting your decision to kick him out & raise your children without him, I just want you to understand that Family Law is not so cut & dried; it's a long tiring process with how busy the system is, and Judges only want the facts, emotions do not help you at all no matter how hard it is to push them down when it comes to the courts.
Also, so you understand why I am certain about your claim that your ex does drugs will not equal supervised visitation - my ex's gf (now wife) was charged and convicted of assault with a weapon for beating my then 8 yr old son with the hard plastic tubing from a vacuum for wetting his pants. My ex was not there at the time, but arrived home within an hour of it happening and was made fully aware of it. His response was to tell the kids not to tell Mommy or I would never let them see him ever again, and then they disposed of the weapon she used.
This happened on a Thursday afternoon, he brought the kids back to me on Saturday morning and no-one said a word; I had no idea something had happened until Sunday evening when I was getting my 8 yr old started for his bath before bed. To this day, I can still vividly see the massive bruises that were on his left side, across his buttocks, across his lower back, and it takes my breath away.
He might not have yielded the weapon, but he knew she broke the law and did nothing to protect his child; the police did not charge him with anything, and CAS did not recommend supervised access, there was a restraining order on her.
So no, you're not going to get an order for supervised access due to 'recreational pot use', sorry.
If you have questions, you can certainly PM me.
Best of luck to you.

