Rav Avigdor Miller on Making Small Talk

Q: The Rav mentioned the importance of speaking as little as possible in order to concentrate our energies on the business of life, which is shleimus, perfection. Making small talk with a friend for a few minutes, is that also forbidden?