If you look at it in terms of how many stat points each core bonus is worth, DMG is easily the best offensive stat unless you rely mainly on DoTs, and DEF is usually (though not always) the best defensive stat.
+1%DMG is nearly 3 stat points' worth (2.86, to be exact). You would need about 71% crit rate from gears/base stats alone (excluding stat points in CRIT and other %CRIT increases) for +1%CRIT to be worth the same number of stat points. In order for +1%SPD to be worth 2.86 stat points, you would need 86% coming from gear/base stats, which is impossible with a 50% hardcap on SPD. The only time the CRIT core would be better is if a significant portion of your damage output comes from DoTs, as the DMG core doesn't affect DoT damage.
+1%HP is less than 2 stat points' worth (1.72 points). DEF beats this at 34% from gears/base stats, which is incredibly easy to get from level 50 green armor. EVA beats it at 60% from gears/base stats (remember to exclude other %EVA bonuses), which means running a full EVA gearset can make the EVA cores worth more than DEF cores. The thing is, if you're running that full EVA set, then your gears are probably giving you closer to 40% DEF (more if you fortify), which means DEF cores break even with EVA cores at 70% EVA from gear/base stats.
Even if DMG/DEF aren't the stats you're focusing on, they're always useful (outside of DoTs for DMG), and having more of them means that any points you would have invested in those stats to optimize your build can just be shifted to other stats. (*Sidenote: with the exception of capping EVA or SPD, it is almost always better to split stats than to dump everything into one stat.)