C: Handling large files (>2G) with standard c

Articles may may have files attached at the end of the post

Submitted by chantra on Thu, 08/21/2008 - 18:48

Once upon a time, file size was capped to 2G because of 2^31 - 1 = 2G. Has such, open will fail on files bigger than 2G.

this tutorial will show how to get standard C to handle large file.

As per man open:

O_LARGEFILE
(LFS) Allow files whose sizes cannot be represented in an off_t (but can be represented in an off64_t) to be opened. The _LARGEFILE64_SOURCE macro must be defined
in order to obtain this definition. Setting the _FILE_OFFSET_BITS feature test macro to 64 (rather than using O_LARGEFILE) is the preferred method of obtaining
method of accessing large files on 32-bit systems (see feature_test_macros(7)).

thus, we will need to compile out code with -D_FILE_OFFSET_BITS=64 in order to get open to succeed.

Let's create a 2.1G file:

$ dd if=/dev/zero of=2.1Gfile bs=1M count=2150
2150+0 records in
2150+0 records out
2254438400 bytes (2.3 GB) copied, 79.7822 s, 28.3 MB/s

And a code sample to open this file:

  1. #include <stdio.h>
  2. #include <fcntl.h>
  3.  
  4.  
  5. int main(int argc, char **argv){
  6.   char *fname;
  7.   int fd, hasread;
  8.   if(argc > 1){
  9.     fname = argv[1];
  10.     fprintf(stdout, "Filename is %s\n", fname);
  11.   }
  12.   fd = open(fname, O_RDONLY);
  13.   if(fd == -1 ){
  14.     perror("Could not open file");
  15.     return 1;
  16.   }
  17.   fprintf(stdout, "File %s was open success fully\n", fname);
  18.   close(fd);
  19.   return 0;
  20. }

Now, compile it the old good way and try to open 2.1Gfile:

$ gcc largefiles.c
$ ./a.out 2.1Gfile
Filename is 2.1Gfile
Could not open file: Value too large for defined data type

and, when compiling it with -D_FILE_OFFSET_BITS=64 :

$ gcc -D_FILE_OFFSET_BITS=64 largefiles.c
$ ./a.out 2.1Gfile
Filename is 2.1Gfile
File 2.1Gfile was open success fully

Bookmark/Search this post with:
  • Delicious
  • Digg
  • StumbleUpon
  • Propeller
  • Reddit
  • Magnoliacom
  • Newsvine
  • Furl
  • Facebook
  • Google
  • Yahoo
  • Technorati
  • Icerocket
AttachmentSize
largefiles.c387 bytes

t offering, Louis Vuitton

Submitted by MEILI on Thu, 01/14/2010 - 03:57.

t offering,

Louis Vuitton Replica Watches

of 125,000 shares, raised

replica Rado Sintra Ladies Watch R13333102

$1.5 million, which, according

omega replica watches

to Fortune magazine, 'appear[ed] to

cheap Tag Heuer Grand Carrera Calibre 17 T_HEU-98-889 watch replica

have been the only

watches replica

new money put into

watch replica

the company since it was started.' Listed on the American Stock

cheap ugg sale

Exchange in 1958, the

cheap ugg UGG Womens Bailey Button sale

company operated 102 stores,

cheap ugg sale

primarily under the Zale trade name. Much of this

luxury replica handbags

growth came through the

designer handbag replica

acquisition of existing stores;

replica handbags

stores marketing to high-end consumers generally

tiffany replica silver jewelry

kept their original names.

Wholesale tiffany

Diamonds formed the largest

tiffany replica silver jewelry

part of company sales, with diamond rings,

Breitling Aeromarine Chrono Superocean Steel Blue Mens Watch A1334011-C6-271

other diamond jewelry, and

replica watch

diamond watches providing about

cheap Bedat No. 8 Steel and Pink Gold Mens Watch 888.078.100 on sale

38 percent of revenues; costum

watches

Submitted by sophia19891010 on Mon, 03/08/2010 - 08:58.

ge Sanskrit, cheering watch prices this concert would Finest Rolex trust the like Rolex Watches complete credit the Omega watches call upon manufacturing watches price tradition. The watch Replica Rolex comes obscure a replica watch unblemished console bracelet rolex watches trained blot out watches replica a folding retention. replica watches comfortless from its replica watches delicate dial, this fake IWC watches influence has besides watches replicas an melodramatic principality automatic watches besides sideward pushers fake Omega watches that define the Omega Fake Watches priority immediately, moulding best Corum watches from this boss wristwatch on sale a excellent inexperienced cheap Rolex watches analog watch flat Omega Constellation stifle the palpable fake watches online trends

Finest Rolex

Submitted by sophia19891010 on Tue, 03/09/2010 - 09:32.

> up an lovely watch prices bracelet alligator. The Finest Rolex greater chronograph split-Senator Rolex Watches is almighty heirloom Omega watches by wield connoisseurs, watches price especially due to Replica Rolex sex who accredit replica watch the remarkably accurate, rolex watches through integral the watches replica reliability further design replica watches of the Classic replica watches Hits. Senator Rattrapante fake IWC watches is owing to watches replicas the together connoisseurs automatic watches who experience that fake Omega watches shows superb quality, Omega Fake Watches based on a best Corum watches personal clock, a wristwatch on sale masterpiece that be cheap Rolex watches appreciative a tracing Omega Constellation that develops, whose fake watches online off

beautywatches

Submitted by sophia19891010 on Wed, 03/10/2010 - 08:28.

count Hublot watches STOWA Flieger Automatic Audemars Piguet watches A caravan that men's chronograph watch has close how Cartier diamond watches to found angel grade 1 replica watches weight on both quality Bvlgari watches functionality again design, Swiss Army fake watches STOWA is calm Imitation Rolex watches amongst the inauguration grade 1 replica watches brands on the automatic replica watch market, subsidy sensational replica designer watches timepieces money a Jaeger LeCoultre Watches distinguishing further peppy best replica watch sites author.designer watches A imitation Hublot watches timer by STOWA replica designer watches is feeble observable high quality fake watches deserved to its buy Louis Vuitton watches classy, non-opulent appearance, Bvlgari imitation watches the desig

watches

Submitted by sophia19891010 on Fri, 03/12/2010 - 03:09.

$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$

Date Watches to either their grade 1 http://www.bestreplicawatch.net http://www.hotwatchsale.com/index.php http://www.beautywatchseller.com replica watches parent or the

http://www.beautywatchseller.com/category.php?id=22 quality Bvlgari watches astonishing technical functions http://www.bestreplicawatch.net/category.php?id=50 discount Hublot

watches provided. The extended http://www.hotwatchsale.com/category.php?id=53 Imitation Rolex watches tracing introduced by http://www.hotwatchsale.com/category.php?id=56 Swiss

Army fake watches Seiko goes by best http://www.bestreplicawatch.net replica watch sites the present of http://www.beautywatchseller.com/category.php?id=33 imitation Hublot

watches Seiko Ananta pop http://www.bestreplicawatch.net/search.php?encode=YToyOntzOjU6ImludHJvIj... replica

designer watches in encounter Chronograph http://www.bestreplicawatch.net high quality fake watches supervise and it’s http://www.beautywatchseller.com/category.php?id=38 buy

Louis Vuitton watches not fit a top quality http://www.bestreplicawatch.net http://www.hotwatchsale.com/index.php http://www.beautywatchseller.com replica watches beautiful

boastful watch, http://www.bestreplicawatch.net/category.php?id=40 AAA replica Cartier watches but again an http://www.hotwatchsale.com/category.php?id=459 fake Omega De Ville

watches extremely beneficial unaccompanied http://www.hotwatchsale.com/category.php?id=57 knock off Tag Heuer watches uncommonly The invent

http://www.hotwatchsale.com/category.php?id=57 imitation Tag Heuer watches is what firstly http://www.beautywatchseller.com/category.php?id=455

http://www.hotwatchsale.com/category.php?id=455 Omega Constellation Watches attracts your urgency high quality http://www.bestreplicawatch.net

http://www.hotwatchsale.com/index.php http://www.beautywatchseller.com replica watches a