Marxists liked to condemn market liberalism. They called the lack of centralized control of the market economic anarchism. The socialists of the soul, our modern day conservatives, condemn social liberalism. The lack of centralized state control of morality, they argue, will lead to license.
But what is it that prevents liberty from turning into license?
First, social liberalism is merely liberalism applied to social issues and liberalism never argued that anyone is free to do anything, anywhere, anytime they please. Liberalism has also had important provisions, often ignored by conservative critics. Liberalism says that the freedom of each man is restricted by the equal liberty, or equal rights, of others. Herbert Spencer said that the “liberty of each, limited by the like liberty of all, is the rule in conformity with which society must be organized.”
But what does it mean? Thomas Jefferson wanted: "A wise and frugal government which shall restrain men from injuring one another, which shall leave them otherwise free to regulate their own pursuits of industry and improvement, and shall not take from the mouths of labor the bread it has earned—this is the sum of good government."
The first limit to liberty is the rights of others. No one may injure another. No one may justly deprive another of their rights to their life, liberty or property.