The simple answer is yes, you can.
On iPad, iPhone, iPod
To avoid looking at the other person simply press the home button. This will allow you to use all your other apps, perhaps to look up something on the internet, while at the same time continuing your conversation.
You will notice the top bar is green while you are doing this, touch the green bar and you go back to the Facetime app and continue your conversation with video.
While you are doing this the other person will se the last image sent with a dark tint over it and a pause symbol - see image.
To stop your image being transmitted is a bit more low key, you could use the back camera - if you have a magnetic smart cover the back camera will show nothing but black, if you have no cover the video will be of your feet or the far wall, or whatever is over there. Or put your finger over the lens! (Or use a Post-it® note – see below)
iMac, Mac Book
The arrangement is different on these devices as they don't have a pause function as the iOS devices do. If you use a different app the FT (facetime) app keeps it's window open and transmitting. Low tech solution, put a sticker over the lens! (Post-it® type notes are recommended as their glue is peelable.) If, for some reason your device doesn't have a camera (i.e. It's broken, or you have an early computer before they became standard - see the instructions below for Mac pro).
Mac Pro and all Mac's without built in cameras.
FT will not allow a connection if no camera is available, so you'll have to attach a webcam to make a call, it's much easier in this case to point the camera away from you or just put a Post-it® note over the lens.