Excerpt from an article by Kyle Orland:

Could Microsoft really have kept a potentially game-changing feature like widespread Windows 8 app support under wraps for this long? A Microsoft spokesperson was incredibly terse and vague in responding directly to the Dell listing, telling Ars in an e-mailed statement that "the suggestion that all Windows 8 apps run on Xbox One is not accurate." (Yes, that is the entire response.)

The idea is fascinating and could only really be feasible if you somehow integrated your Xbox One interface with a touch centered device (I doubt Kinect is the answer). In reality the clear choice for this kind of TV interface should be Xbox SmartGlass, however, its current iteration left a lot to be desired.