Yes. You're describing includes here. The three most common forms are PHP, ASP and SSI. All you have to do is place the content you want shown on every page into its own document. You can even give that document any file extension of your choice.
Then a small snippet of code goes into your HTML document:
<!-- #include virtual="PATH/FILENAME.EXT" --> for asp
<?php include ("PATH/FILENAME.EXT"); ?> for PHP
I'd keep away from SSI if you have servers that can handle either of the above.
Rename your HTML document from FILENAME.html to FILENAME.php or FILENAME.asp
That's all there is to it. It's frighteningly easy