Is it possible to have a finitely axiomatizable arithmetic in first-order logic?
What I say in the following may be completely wrong, sorry about that.
NBG set theory, is as powerful as ZFC set theory, as they can both prove the same set of theorems. On the other hand, Peano's arithmetic is simpler, but it is also less powerful (e.g., Kirby-Paris hydra's theorem cannot be proven in Peano's arithmetic).
So informally, in terms of theory power (number of theorems that are provable in said theory), we have NBG = ZFC > Peano.
ZFC and Peano are not finitely axiomatizable, as they rely on axiom schemas, that create an infinity of axioms in first-order logic. However, NBG is finitely axiomatizable, as it contains no schema of axioms in first-order logic.
So considering the power relations between all these theories, would it be possible to "emulate" an arithmetic theory with a finite number of axioms using NBG? If so, would it be possible to create a "new" arithmetic theory (not necessarily as powerful as set theories) with a finite number of axioms in first-order logic?
First thing to point out is that there are many theories of first order arithmetic which are weaker than $\textbf{PA}$ which are also finitely axiomatizable. For example Robinson Arithmetic $\textbf{Q}$ and $\textbf{I}\Sigma^0_n$ which is $\textbf{PA}$ with induction restricted to $\Sigma^0_n$ formulas. If you want the theory to extend $\textbf{PA}$ and remain in the same language I believe you are out of luck. That is, there are not finitely axiomatizable extensions of $\textbf{PA}$ in the language of first order arithmetic $\mathcal{L}_1=(0,S,+,\cdot,<)$. I don't have Kaye's book at hand at the moment but I will add the reference to this answer when I have it. The idea is any axiomatization of an extension of $\textbf{PA}$ cannot have axioms of bounded complexity for partial truth predicate reasons.
But as $NBG$ is a "second order" extension of $ZFC$ there is also a "second order" extension of $\textbf{PA}$ called $\textbf{ACA}_0$ which is finitely axiomatizable and is conservative over $\textbf{PA}$. Just as in $NBG$ we extend the language to be able to talk about classes, we extend the language of arithmetic to speak about sets of numbers. We of course cannot work with second order semantics otherwise we will find ourselves with a categorical theory (See Henkin Semantics in the Wikipedia article: https://en.wikipedia.org/wiki/Second-order_logic).
Edit: The way we change extend the language of first order arithmetic $\mathcal{L}_1$ to that of second order arithmetic is to add a new sort of objects, namely that of sets of numbers. We use the convention that number variable are denoted by lower case letters $x,y,z,w,\dots$ and second order variables with capital letters $X,Y,Z,\dots$. We add a new symbol $\in$ to the language which is that of a binary predicate which accepts as first entry a number term and in the second entry a set term. We refer to this language as $\mathcal{L}_2$ or the language of second order arithmetic.
$\textbf{ACA}_0$ can be axiomatized as an $\mathcal{L}_2$ theory having the same axioms as $\textbf{PA}$ for numbers, and for each formula $\varphi$ without set quantifiers an axiom which states that the set $\{x:\varphi(x)\}$ exists, this is called the arithmetic comprehension axiom scheme. The reason why $\textbf{ACA}_0$ is finitely axiomatizable is essentially because of Post's theorem, one can replace arithmetic comprehension by the axiom stating that the turing jump of every set exists and the induction axiom by the set induction axioms. Details might be in Simpson's Subsystems of Second order arithmetic and in Hájek and Pudlák's Metamathematics of Second Order Arithmetic.