there's not as big a leap in power between the two consoles.
In terms of the PS4 CPU, no, it's weaker but a more standard design, however, in terms of the GPU and the RAM bandwidth on hand the PS4 is a massive leap over the PS3. PS2 had more bandwidth to its VRAM than the PS3 did. Anyway, you are comparing apples and oranges here.
You can in theory run any PS4 game on a PS3 or any PS3 game on a PS2, but it will not be the same game as it would require serious reworking or re-writing from scratch, or maybe even totally redesigning. Just as the PS4 has 8GB of RAM vs the PS3's 512MB, the PS3 has 512MB vs the PS2's 36MB - Which one is the bigger jump here (Ok, I'm cheating, I didn't include the 2MB inside the Cell)? Is it just as difficult to squash 8GB into 512MB as it is 512MB into 36MB?
That graph is not showing real number performance figures but rather relative theoretical performance figures in x# for the GPUs in these systems with the WiiU as the baseline of 1 (Prim is short for primitive).
Regarding you point as a whole though, yes the PS3 is quite powerful hardware handicapped in relation to modern hardware more by its RAM bandwidth and amount more than anything. Sony could easily have tweaked the PS3 to give XBone level performance, but they chose not to, in part due to what Cell brought to the table being both expensive and no longer required - modern GPUs don't need the helping hand.
In any case, I don't think that Sony released any of its PS3 titles for the PS2 apart from games such as MLB The Show, which this year is releasing on PS4, PS3, and PS VITA in a similar way - even the Sony F1 game on PS3 didn't make it to the PS2, and funilly enough the PS3 F1 game was released before GOWII on the PS2, the PS3 remastered HD edition of which did not land on the PS3 until 2 and a half years later. Third parties are in a different position however, they do not have a hardware platform to push and will chase the market they can sell more games to, and as long as people owning the older platforms continue to buy the games in sufficient numbers they will release their games for those platforms in addition to the newer ones.
It's called business! Take FIFA 14 on iOS and Android as an example. WTF is it doing on those devices? How the feck do you control the fecking thing? On PS VITA it uses both sticks, square, cross, circle, triangle, R1 and L1, as well as the front and back touch gimmicks if you want to turn them on - it's 1.3GB on Google Play as well vs the VITA 2.9GB on the PS Store and the 9 or so GB on the PS4.
Looking at the UK charts the PS4 in the main is outselling the PS3 in terms of software most of the time, but as the PS3 sales in proportion are still relative good then third parties will continue to release their games multi-platform. This who argument is a mirror of people with gaming PCs whining that their version of the game has been gimped because it was developed for console. Well, guess what? The console versions sell better so the publishers focus more on them than the bloke with a 1500 quid PC.