HTML is the bracketed text that makes up web pages (e.g. "<html><head><title>hi</title></head><body>sup</body></html>"). Each bit of bracketed text is called a tag. There is a rigorous definition of what the valid tags are in HTML, as well as how they relate with each other. Each version of HTML (e.g. HTML5, the current standard-bearer) adds new valid tags and deprecates others. AMP is a specification that only allows for certain HTML tags, while adding support for others that are not expressly part of the HTML specification. The functionality of these tags is provided in the AMP JS.
In normal HTML, you specify an image with the <img src="..."> tag. AMP uses the <amp-img src="..."> tag instead. The reason is that the amp-img tag also requires the width and height, whereas in HTML - for the img tag - the width and height are optional and the image could be sized and layout refit after the image is loaded. AMP needs to be able to calculate the entire page's layout on its initial load - this is fundamental to AMP's specification. So the tag is changed to require size upfront. Video tags are similarly changed to ensure the page quickly upfront.
CSS (cascading style sheets) specifies the look and feel of a web site. Standard HTML allows for CSS to be defined in external files that can allow for sizes that are variable based on a number of attributes (e.g. the sizing of the browser window). AMP requires all elements have a predefined size from the beginning - and that the CSS be specified all in a single file in the main page. This reduces the number of calls required and also provides a very fast mechanism to determine the sizing and location of all elements in the page.
Caching is the third pillar of AMP. To ensure the initial rendering happens quickly, the content can be delivered not from a normal web server, but from an optimized content delivery network. The initial page load is routed to a physically close server optimized for speedy delivery. Google and Cloudflare offer AMP-optimized caching. It is not required that AMP content be delivered from a cache, but, for example, the Google Search AMP results will only be those results that live on the Google or Cloudflare Cache. As Google says, "by using the AMP format, content producers are making the content in AMP files available to be cached by third parties. For example, Google products use the Google AMP Cache to serve AMP content as fast as possible."