The term smallfolk refers to the peasantry and common folk of Westeros, effectively anyone who is not part of a noble house (though smallfolk make up much of a noble household) or a knight (though hedge knights are generally considered to be only one tiny step above smallfolk).
In the books
Smallfolk make up the common working and middle classes of Westeros, serving as farmers, labourers, builders, shopkeepers, servants and soldiers. They are ruled over by their lords, but are not slaves, slavery being abolished in Westeros thousands of years ago at the command of the Faith of the Seven.
Because of their lack of authority and power, smallfolk tend to be neglected by their lords, seen only as a suppliers of food or soldiers. Smallfolk tend to look up to knights and religious folk who stand up for their interests.