Replay	Ndx
	Home | Changes | Index | Search | Go

NDX files

Introduction

Overall Structure

Vagueness

Notation

Header

struct ndx_header {
    u8 major_version;    /* 2 */
    u8 minor_version;    /* 2 */
    u8 flags;            /* 0x01 = copy protected; none others seen */
    u8 unused[29];       /* all 0s */
};

Record

struct ndx_record
{
    u8  flag_1;         /* all unknown; different between 520410600 and
                           520411140.  Update: looked like a single flag in
                           520410600, but appears unused & uninitialized in
                           520411140 and the very first software release */
    u8  comm_flag;      /* 0x01 == part of a commercial break; 0x02 ==
                           candidate to be the beginning or ending of a
                           commercial break */
    u16 video_offset;   /* offset, relative to the stream_position member, of
                           the start of the first video PES packet in this
                           pack */
    u8  unk_fe;         /* always 0xfe as far as I've seen; I think this is
                           read directly from one of the mpeg encoder's
                           registers, but I don't know what it means */
    u8  macrovision;    /* always 0 or 3 as far as I've seen; I think this too
                           is read directly from one of the mpeg encoder's
                           registers, and '3' means 'macrovision possibly
                           detected' */
    u16 macrovision_count; /* counts the number of consecutive frames with
                           macrovision fields equal to 3; I think if this
                           reaches 361 (more than 3 consecutive minutes), then
                           the 'copy-protected' flag is set -- details on when the flag is set
                           have changed in 4.1.1 */
    u32 audio_offset;   /* offset, relative to the video PES pointed at by
                           video_offset, of the beginning of the first audio
                           PES in the pack */
    u32 unused;         /* always 0; related somehow to the timestamp, it
                           looks like */
    u64 timestamp;      /* timestamp in nanoseconds, from an unknown base,
                           possibly system startup time.  What *exactly* it's
                           a timestamp of is still somewhat in question; it
                           doesn't match up with any of the timestamps in the
                           MPEG file.  I think it's probably the time the
                           record was written, as measured by a clock that
                           isn't exactly synchronous with the MPEG encoder's
                           clock */
    u64 stream_position; /* the offset into the MPEG file of the start of the
                           32k cluster the pack this record is about starts
                           in. */
};

Bugs

-- ToddLarason - 09 Mar 2002

There is also a possibility that the .ndx is correct and the .mpg has errors in it.

-- LeeThompson - 01 Sep 2002

NOTE: Replay5000s have a different version: 3 and 0. ndxdump freaks out a lot so I'd say that the format has changed significantly for Replay5000s.

These seem totally different on the 5000. Right now I'm guessing that the header is much larger judging form the amount of duplicate information between different .NDX files. I'm guessing 40-48 bytes.

-- LeeThompson - 23 Dec 2002

I have a 5040 and have looked at a couple of .NDX files for some 30 minute shows.

The first 32 bytes of the .NDX file is unknown to me as of yet.

However, I have found that everything after that seems to be in 24 byte records. Like so:

struct ndx5000_record {
  u64 timestamp;    /* 8 byte timestamp, in nanoseconds */
  u32 empty1;       /* apparently empty(?)
                       could be part of the video packet offset,
                       to accomodate some insanely large files */
  u32 video_offset; /* video stream packet offset */
  u32 video_length; /* video stream packet length(?) */
  u32 empty2;       /* apparently empty(?) */
};

Those two empty 32-bit words could be used for something else. Like I said earlier, I only analyzed one or two .NDX files.

Hope that helps.

-- TimLee - 13 Jan 2003

You're farther along than I am. About all I'm sure about is BYTE 1 and BYTE 2 of the header is some kind of version number. :)

So you're thinking this has the MPEG packet offsets in it for some reason? Damn I wish I knew what they were using this file for now.

(I'll admit right here, the new NDX file just confuses me. :) )

-- LeeThompson - 15 Jan 2003

Hmm your index_record is 24 bytes not 16 :)

I think the header is 40 bytes or so... have no idea what the entries actually are so it's a mass of unknowns.

-- LeeThompson - 15 Jan 2003

I'd not be surprised if it's all empty except the version, and is sized so that it'll be read as otherwise-all-0s by the ndx readers for all prvious versions.

-- ToddLarason - 15 Jan 2003

Oh it's not empty. Just don't know what the numbers are used for.

-- LeeThompson - 15 Jan 2003

oh. I should really get my obsession turned back on and look at some of these files again...

-- ToddLarason - 15 Jan 2003

Oops... I guess I need to work on my math :-(

-- TimLee - 15 Jan 2003

I hadn't seen any of this work, I'm glad I found it. The other morning at 6am I took a look and what I found. (I don't have notes on me, but what I remember is:

Header size is 14 bytes, after that blocks of size 48 bytes. The nano counter took about 3 seconds to find.

I'll look at the data again now that I see what's been posted here.

-- TWikiGuest - 14 Feb 2003

TimLee? is correct, the header is 16 bytes, each record is 24 bytes, I believe empty1 is actually part of the video_offset (making it 64 bits). video_position would be a better name for it. I guess I'll have to record a > 4Gb show to verify. video_length is actually audio_offset but defined as an offset from video_position. empty2 is likely to hold things like the commercial flag but I cannot tell yet. Anyway, hope this help...

-- TWikiGuest - 29 May 2003

ReplayTV 5000 Series NDX Files

struct ndx5000_record {
  u64 timestamp;            /* 8 byte timestamp, in nanoseconds */
  u64 filepos_iframe;       /* File position to an I-frame */
  u32 iframe_size;         /* Size of the I-Frame (including PES Headers) */
  u32 empty;               /* Always Zero, possibly for alignment */
};

(Thanks to Anon!)

-- LeeThompson - 29 Jun 2003

Is it correct to guess that the 4xxx and 5xxx streaming incompatibility is due (at least in part) to this? Or are the .mpg files also different?

-- TWikiGuest - 28 Jul 2003