You'll have to use a template for that, but an easier way would be to use server-side programming for the common parts of the page and have things like the navigation as a separate file. That way, you only have to update one part of the page, as opposed to updating the template, updating all the pages and then uploading all the new pages again.
You change the template and then it'll ask you if you want to change all the pages that use that template. You click "yes" and it'll change all the pages. Once the pages are changed, then you have to upload them all to the site again.