First, let me start by saying I had a lot of these same questions with each expansion release after the initial Core starter set. Part of the confusion, I think, stems from the fact that the Core starter set is nigh impossible to find, selling out quickly due to the unbelievably good value of the box contents. Then you have 3 big expansions to digest for a game that is, itself, less than a year old. Concerns about fragmenting a brand new, budding player base are not entirely unwarranted at this stage; admittedly, that's a lot of releases in a short span of time. This is just me guessing, but I think the strong demand for this game at launch maybe caught Games Workshop a little by surprise, and in response to that, they accelerated the release of the expansion box sets so there would be product on shelves for the holidays. When Rogue Trader (RT) dropped, there was a lot of chatter about how this was supposed to come after the Commanders release in the original product road map, but with no more supply of the big core starter set I can see why they would rush the release of the more expensive RT box.
Anyway, back on topic. What do you actually need to play the game? Simply, the Core Manual.
If you already have miniatures, then this is all you need. If you need or want a new team, there's an ever-growing selection of great team starter sets that will come with all the faction specific tokens & tactic cards, not to mention some great discounted terrain pieces to boot. A cache of bitz is also nice to have as the game encourages customizing / kit-bashing to make your teams unique, especially to represent the Specialist models.
Honestly, if you're coming from 40k (and a lot of us are), yes the core manual is a necessity as it contains all the rules* and points for all the factions**, but it gives you way more than that and as someone who really enjoys list building & theory-hammering, it's a source of countless hours of entertainment on its own. Absolutely worth the $40 and such good value that it makes you wonder if the smaller, more expensive expansion books are actually worth the investment. What do the expansions do?
Kill Team: Commanders
This expansion is the obvious one. Like the name suggests, it provides rules for playing with Commanders in Kill Team. This is entirely optional and the important thing to note here is that in order to use them, you must play a mission that allows them (which of course the book provides these). But the distinction here is that these are not standard or "vanilla" Kill Team missions. The points limits are typically higher so you can actually fit them into your list, aka Command Roster.
I'll be honest, when I heard the news of this expansion it actually worried me. It seemed to come so quickly on the heels of the core / standard ruleset, I was concerned the core manual was about to be invalidated before the masses even had a chance to enjoy the basic game, which is largely contained to troops vs troops -- that may seem boring to some, until you try it. In my opinion, one of the best strengths of vanilla Kill Team is reacquainting yourself with what the standard troopers can do and accomplish on the table. It's easy to overlook them in 40k alongside giant war machines and shiny elites, but troops are what win you games. Adding a big badass HQ to this mix throws a big wrench into the balance of what Kill Team seems to be all about; lowly troop on troop skirmishes. There's also a lot of narrative appeal in leveling up your average joe (hey!) trooper based on the heroic actions that inevitably occur in the standard game.
I admit, it's not as bad as my initial gut feeling and I totally understand that some love hero-hammer, even I do from time to time; but that's just it, it gives another layer and way to play that is entirely optional, take it or leave it. I don't expect to see many events or tournaments allow them or select missions that allow them, but who knows.
Kill Team: Arena
This is the latest expansion, and yes I know I'm not following the release order of these but bear with me for a moment. The way I see Arena, it's basically a slightly stripped down, more rigid version of standard vanilla Kill Team. The biggest thing to note about this variation is that all terrain and warzones (the actual map you play on) are all explicitly prescribed in the Arena book/box -- and they all take place indoors. Perhaps "Kill Team: Indoor Arena" or "Kill Team: 2D" would be a more descriptive name of the types of missions this book presents. I don't say that as a good or bad thing, but understand that this version of the game uses none of the beautiful and amazing terrain that you get in the core or team starter sets, or any existing terrain you may already have. This could be off-putting for some, but this is all done by design. These indoor/arena maps are completely symmetrical with limited, identical terrain, and missions & objectives that are intended to be as balanced as possible. Very much along the lines of the NOVA Kill Team mission pack, but no three dimensional terrain.
The gamer in me likes the design philosophy of Arena, as it strives to be not only fair but fast to play. This is the version of the game you can play on your lunch break at work, and that sounds awesome to me. For some, this variant may lack the spectacle that a densely packed tabletop of gorgeous terrain ignites. But again, like Commanders, Arena just offers yet another way to play the great game that is Kill Team.
The gamer in me likes the design philosophy of Arena, as it strives to be not only fair but fast to play. This is the version of the game you can play on your lunch break at work, and that sounds awesome to me. For some, this variant may lack the spectacle that a densely packed tabletop of gorgeous terrain ignites. But again, like Commanders, Arena just offers yet another way to play the great game that is Kill Team.
Kill Team: Rogue Trader
This one is interesting for a lot of different reasons. It immediately draws the attention of us older 40k gamers, donning the namesake of the beloved 1st edition of Warhammer 40,000. You get rules for playing all the new models in 40k as well which is a nice touch. Nostalgia aside, I was immediately drawn to the awesome Truehawk (indoor) killzone board. But clocking in at $30*** more than the amazing core starter set (yet no where near the amount of terrain/plastic), is it worth it?
***Anecdotally, I've seen this set heavily discounted in local shops; like well below even the starter set, and unlike that one, seems to be plenty in stock...
Truth be told, even if you pay full retail I believe this expansion set is worth it. Here's why. This is 'the one expansion set to rule them all.' Hearkening back to the beginning of this long post, and after investigating the 2 expansions above, I can totally see how this particular expansion set was not supposed to be the first of these 3 official Kill Team expansions to be released. Rules for fighting indoor/close confines a la Kill Team: Arena are in the Rogue Trader book almost identically word for word (you roll off to open doors, which I think is better & more clear than Arena's 5+ roll for the player attempting). Arena's secondary objectives and competitive / tournament play supplemental rules are almost identical to what you'll find in the freely available NOVA Kill Team mission pack (I link to this in my NOVA KT Tournament Recap).
The Rogue Trader book also contains some Kill Team: Commanders rules, enough to get the gist of, as this set "introduced" the first 2 commanders to the game. In my view, this set is the best of both worlds, along with entirely new models (which I think are brilliant) and can also be used in 40k proper, an awesome spaceship killzone with the other side very similar to the other indoor arena zones, and a great narrative to boot. Commanders + Arena clocks in at ~$150 combined, so you save $20 forgoing those individually and opting for Rogue Trader. Sure, there's more to the full fledged Commanders book and Arena comes with 2 killzone boards and extra cards, but even still I feel Rogue Trader better implements both aspects of those expansions and offers way more in terms of overall value.
Lastly, I present you with my own home-brew expansion...
Kill Team: Space Hulk
If you happen to have a copy of the fantastic GW classic, Space Hulk, you can make your own indoor layouts (or to represent underwater tunnels -- I love that Dreadfleet water mat!). I've been teaching and playing Kill Team with my 7 year old, borrowing elements from all of the above to form a campaign of our own creation. I think this has been the most fun variation of the game for me personally :)
TL;DR - whichever way you go with Kill Team, there's good fun to be had. But by all means, if you find a copy of the Core starter set, you know what to do ;)
almost forgot about these...
*all the rules for organized play anyway; specialists are capped at level 1 abilities
**all factions prior to Rogue Trader, but all the ones 40k folks are familiar with
Cheers!