Make a rule to always add an issue first and then refer it with #number in your commit message. It very easy to ignore this thing. For every small/big change adding a reference might look like extra work. But doing so is really worth.
The real benefit of adding references to commit messages becomes apparent when you are looking for all the changes that were done for particular feature. If you are developing on Github or using project management software like Redmine, these references becomes links to the issue.