Nice your code works pretty well!
Well we have downloaded around 100,000 articles from Nexis, it only allows saving 100 per file so we have quite a lot of files. We want to reduce these and will probably have around 20 categories these will fit into. So in the end we want to have 20 text files.
It is not entirely clear if the analysis will be done in R or in Python, therefore in the 1st step we want to format them uniformly and assign them IDs. (Historically one of the researchers in our group has even used SAS for a similar task and would like the .txt files as well). This allows us to keep more flexibility.
Text structure should be as follows (this is an example, all this data except the ID in the 2nd row and the two ** after the end of the article are included in the original article):
Chrysler will launch ads for the Dodge Caravan that target Asians
Title: Chrysler steps up Asian-language ads in Western markets
Source: Automotive News, 76 : 8, December 31, 2001. ISSN: 0005-1551
Publisher: Crain Communications Inc.
Document Type: Journal
Record Type: Fulltext Word Count: 317
Publication Country: United States, Language: English
By: Julie Cantwell
The Chrysler group is getting ready to ring in the new year in California.
In celebration of the Chinese New Year in February, Chrysler will introduce the
second part of its advertising campaign to Asians, adding the San Francisco
The campaign began this fall in Los Angeles with a 30-second TV spot and
newspaper ads promoting the Jeep Liberty in Cantonese and Mandarin. On Feb. 4,
the Chrysler group will add the Dodge Caravan to the campaign, with similar
For 2002, the Chrysler group will increase marketing spending to reach Asians
and Hispanics in the West, said Steve Shugg, director of the Chrysler group's
West business center. He would not provide details.
It's more of a brand-building thing,'' Shugg said. The intent is not to sell
cars; it's to provide Hispanic and Asian communities with an experience in a
The Chrysler group has struggled in the West, where imports dominate in car
sales and are gaining in trucks. Its 2001 sales in California through September
were up 1 percent over the same period of last year. September sales were 12.2
percent below the figure for September 2000.
To gain strength in the West, Chrysler group marketers realize they must connect
with Asians and Hispanics on the West Coast.
``This will give us a better understanding of Asian-American consumers and
position the company to gain share in this important market,'' said Jeff Bell,
Chrysler group vice president of marketing communications.
Chrysler is trying to reach California's Asians through events as well. In
November, Chrysler for the first time offered a couple of its longstanding
events, Jeep 101 and Chrysler Proving Grounds, in Spanish and Asian languages.
The next Asian-language Jeep 101 is scheduled for Feb. 3 and 4 at the Asian
American Expo in Pomona, Calif.
Imada Wong Communications Group Inc. in Los Angeles is handling the Asian
campaign. Chrysler considers the campaign a pilot program for future attempts to
Copyright 2001 Crain Communications Inc.
Brand Names: Dodge Caravan; Jeep Liberty
Company Names: CHRYSLER CORP
Concept Terms: All market information; Asian American market; Hispanic market;
Geographic Area: North America(NOAX); United States(USA)
Industry Names: Automotive
Marketing Terms: All campaign; All media; All product marketing; New campaign;
Newspaper advertising; Positioning-repositioning; TV advertising
Product Names: Passenger cars(371100); Minivans(371188); Sports utility
vehicles, 4-wheel drive(371177)