Friday, September 9, 2016

The First Domestic Mass Data Mining Systems

Mass data mining first was developed by the Federal Bureau of Investigation (FBI) during the late 1990s and supervised by its Data Intercept Technology Unit. The first program, whose name remains a secret, debuted in 1996 and was replaced by the programs known as Omnivore in 1997 and Carnivore in 1999. (Carnivore later was renamed DCS-1000.) Carnivore was “packet sniffer” software: It captures the e-mail messages of a specific target from the e-mail traffic that travelled through the network of an Internet service provider (ISP). In 2000, when the program was publicly disclosed, the FBI said it “chews all the data on the network, but it only actually eats the information authorized by the court order.”[i] In another statement of this view, an assistant FBI Director told Congress in 2000: “[Carnivore] does NOT search through the contents of every message and collect those that contain certain key words like ‘bomb’ or ‘drugs.’ It selects messages based on criteria expressly set out in the court order, for example, messages transmitted to or from a particular account or to or from a particular us.”[ii] So before 9/11, the FBI said the program did not conduct mass data mining, although it had that capability.

Carnivore became a mass surveillance tool, bypassing the warrant requirement to search records, sometime after the attacks of September 11, 2001. I have tried to identify when exactly this changeover to mass surveillance for phone and internet records occurred, but it is difficult to pinpoint based on the current state of public knowledge. To be sure, FBI mass data-mining efforts are authorized under the USA Patriot Act (2001) and FBI Guidelines (2002) for investigation issued by the U.S. Justice Department. Carnivore (or DSC- 1000) was used until 2005 and replaced by a program known as NarusInsight. The technology writer Declan McCullagh writes that NarusInsight “can record all Internet traffic, including Web browsing--or, optionally, only certain subsets such as all e-mail messages flowing through the network. Interception typically takes place inside an Internet provider's network at the junction point of a router or network switch.” According to Kevin Bankston, a staff attorney at the Electronic Frontier Foundation, "What they're doing is intercepting everyone and then choosing their targets." This approach is called “full-pipe” surveillance and critics have questioned its legality. [iii]

[i] E. Judson Jennings, “Carnivore: US Government Surveillance of Internet Transmissions,” Virginia Journal of Law and Technology, 6: 10 (Summer 2001). See also Talitha Nabbali and Mark Perry, “Going for the Throat: Carnivore in an Echelon Word- Part I,” Computer Law and Security Report Vol. 19 no. 6 (2003): 460.

[ii] Oversight Hearings on “Fourth Amendment Issues Raised by the FBI’s ‘Carnivore’ Program,” Before the House Committee on the Judiciary, Subcommittee on the Constitution, 106th Cong. (2000) (statement of Donald M. Kerr, Assistant Director, Federal Bureau of Investigation), July 24, 2000 (quoted in Jennings, “Carnivore”).

[iii] Dean McCullagh, “FBI turns to Broad New Wiretap Method,” ZDNeT, January 30, 2007; Richard Koman, “FBI ‘Full-Pipe’ Surveillance May be Illegal,” ZDNet, January 31, 2007.

Wednesday, June 8, 2016

What Does Your "Metadata" Say?

In 2005, Washington Post journalist Robert O’Harrow published a popular book on mass data-mining entitled, No Place to Hide.  He identified new ways both industry and government collect vast amounts of personal information on Americans by working separately and also in collaboration.  The “War on Terror” had accentuated a data-driven surveillance society. The book received widespread notice. The conservative columnist William Safire wrote in the New York Times: “The computer's ability to collect an infinity of data about individuals -- tracking every movement and purchase, assembling facts and traits in a personal dossier, forgetting nothing -- was in place before 9/11. But among the unremarked casualties of that day was a value that Americans once treasured: personal privacy.” The idea that individuals could retain a sphere that is ''nobody's business'' rapidly had disintegrated.  A new “big business of everybody's business” had become the order of the day.
            Meanwhile, liberal law professor Geoffrey R. Stone, after reading O’Harrow, raised an existential question. “Once we understand that our every move is being tracked, monitored, recorded and collated, will we retain our essential sense of individual autonomy and personal dignity?” Where do people retreat if there is no place to hide?  There also are serious risks inherent in the construction of new data-based dossiers: data error; stolen data; and unintended public data disclosure.  Finally, as Stone notes, government may use its data collection capability “to suppress dissent and impose conformity.”  Despite official claims that data-mining promotes security, “history teaches that once government has such information, it will inevitably use it to harass and silence those who question its policies.”
            To be sure, O’Harrow was not the first writer to tackle this important subject matter.  Almost a decade earlier, academic specialists David Lyon and Elia Zureik edited an important book, Computers, Surveillance, and Privacy (1996), in which they had identified the new issue of “dataveillance.” For example, one of the authors in that volume, Colin J. Bennett, wrote:
 Mass dataveillance begins with no a priori knowledge of the individual(s) who may warrant attention. Its aim is to screen groups of people with a view to finding those worth subjecting to personal dataveillance. It is based on a general rather than specific suspicion, but also tries to deter or constrain behavior. All forms of computer matching are mass dataveillance techniques. They all involve the aggregate comparison of different data systems to identify those ‘hits’ that prima facie warrant further investigation.
Today, mass data-mining affects more Americans than ever before.  This is so because electronic records widely have displaced paper records and electronic communications now are prevalent in many spheres of both our personal and public lives.  More than 90 percent of Americans use cell phones.  The Internet has spread across the landscape transcending boundaries of race, gender, and class. In 2013, more than 85 percent of the nation’s population regularly went online.  More than half of the entire American adult population uses online social networking sites.  U.S. authorities also ask online service companies for account information on thousands of individuals.  To some extent, the idea that too much data now exists to make sense of it is relevant.  The common concern -- “drowning in data but starving for knowledge” —poses challenges for government data-mining, but the official development of more efficient systems for record matching and sorting promises to keep pace with the explosion of information.
            Of course, not all data-mining is nefarious. It can be an effective tool for scientists and other researchers, who refer to it as “knowledge extraction” and “information harvesting.”  It builds knowledge from large sets of data by identifying patterns; it makes generalizations about future behavior based on past behavior.  Data-mining can be used for “pattern detection” to identify small departures from the norm, or unusual patterns.  As information analyst Joyce Jackson notes, “Data mining allows the automated discovery of implicit patterns and interesting knowledge that’s hiding in large amounts of data.”
            But while data-mining proves useful in some fields, its application to the “War on Terror” at best is dubious. There is no way that patterns discerned from data analysis can predict political violence.  What may appear to be “suspicious” behaviors or patterns likely are anomalies – an oddity or peculiarity with little discernible meaning.  Using anomalies to create a suspect list is deeply flawed.  As Jim Harper of the Cato Institute concludes:
First, terrorist acts and their precursors are too rare in our society for there to be patterns to find. There simply is no nugget of information to mine.
Second, the lack of suitable patterns means that any algorithm used to turn up supposedly suspicious behavior or suspicious people will yield so many false positives as to make it useless. A list of potential terror suspects generated from pattern analysis would not be sufficiently targeted to justify investigating people on the list.
A major study conducted by the National Research Council confirms this analysis.  The report, ironically funded by the U.S. Department of Homeland Security, offers a blistering attack on the effectiveness of data-mining for terrorism discovery. “Automated identification of terrorists through data mining (or any other known methodology) is neither feasible as an objective nor desirable as a goal of technology development efforts," the report found. "Even in well-managed programs, such tools are likely to return significant rates of false positives, especially if the tools are highly automated." A false positive -- that is, erroneously identifying someone as a terrorist suspect -- can have disastrous consequences for individuals.  It can lead to major privacy intrusions, as well as targeted surveillance and harassment in everyday life if security agencies decide to “neutralize” subjects. False positives can lead to individuals “being in trouble with the government” for no legitimate reason.      
            So the ability of government to sort through mass data to discover preparation and planning for terrorism is a waste of resources. By contrast, data-mining is very effective to identify people and groups involved in dissident politics.  Both the FBI and NSA can sort through billions of records to find patterns of expression critical of government. Once the FBI locates subjects to neutralize, they can use data-mining directed at specific individuals to maximize their intelligence operations.  The National Research Council reports: 

"Once an individual is under strong suspicion of participating in some kind of terrorist activity, it is standard practice to examine that individual’s financial dealings, social networks, and comings and goings to identify coconspirators, for direct surveillance, etc. Data mining can expedite much of this by providing such information as (1) the names of individuals who have been in e‑mail and telephone contact with the person of interest in some recent time period, (2) alternate residences, (3) an individual’s financial withdrawals and deposits, (4) people that have had financial dealings with that individual, and (5) recent places of travel."

Wednesday, April 13, 2016

Some Current Writing Projects

I am working on several writing projects:
  • A graphic novel on the history of surveillance to be published in 2017.
  • Forms of monitoring in George Orwell's Nineteen Eighty-Four.
  • Political policing in an Oligarchy.
  • Some thoughts on "authorization."
  • Government surveillance of ethnic groups in the U.S.

Friday, November 27, 2015

A new book, Facing the Future: Paintings, 2007-2010

I wear several different hats.  Besides being an engaged writer on surveillance, social movements and civil liberties, I am a visual artist (painter).  A new book featuring my art has been published by Cyberwit.  It features work completed from 2007 through 2010.

Thursday, November 12, 2015

Two Book Chapters are Forthcoming

"A White Man in the Colored Bronx" in Teresa A. Booker, ed., Public Space, Public Policy and Public Understanding of Race and Ethnicity in America: An Interdisciplinary Approach, (University of Akron Press, 2016).

"Is Mass Surveillance an Effective Tool in Fighting Terrorism?" in Richard Jackson and Daniela Pisoiu, eds., Contemporary Debates on Terrorism Vol. 2, (Routledge, 2016). 

Sunday, May 31, 2015

I wrote a chapter ("The State Response to Occupy: Surveillance and Suppression") in this new book, What Comes After Occupy? edited by Todd A. Comer

From the Introduction to the book:

The last two essays described above mined the Internet in order to get a better sense of Occupy’s demographic and organizing shortcomings with the ultimate goal of improving future social movements. Ivan Greenberg, in a complementary fashion, in our final essay on “The State Response to Occupy: Surveillance and Suppression” shows in detail how the federal government and its lackeys also mined social networking in order to obstruct radical politics in multiple U.S. cities. He then argues that “[e]stablishing a thick historical record is vital in order to analyze the movement's strengths and limitations . . . Police and intelligence records can add specificity and historical consciousness about what the movement represented to official power and the threat it posed to remake society.” Greenberg ends by arguing that protestors will need, increasingly, to “occupy surveillance” to ensure an effective politics in the future. Surveillance systems are not coherent, seamless tools of power; we need to locate weaknesses, and find ways to undermine surveillance in tactical ways: “Overall,” he writes, “the new reality of the surveillance society is sobering, and the tenacity of the American state to protect its practices should not be underestimated.” Still, he writes, with the advent of Edward Snowden, a critical movement is beginning to take shape to combat excessive government surveillance of local cities.

Friday, March 20, 2015

New article just published, "From Surveillance to Torture: The Evolution of US Interrogation Practices during the War on Terror"


From surveillance to torture: The evolution of US interrogation practices during the War on Terror

Ivan Greenberg
DOI: 10.1057/sj.2015.7
The war on terrorism weakened the distinction between observing suspicious bodies and tortu…
Print Publication Date: 16 Mar 2015

Online Date: 16 Mar 2015

Sunday, December 28, 2014

A new year brings new published writings. Forthcoming publications in 2015.

“From Surveillance to Torture: The Evolution of U.S. Interrogation Practices during the War on Terror,” Security Journal (forthcoming Spring 2015).

 “Proletarian Literature from the Bottom Up: Workers and Poetry during the Rise of the CIO,” American Quarterly (forthcoming June 2015).

“Postage and Power: U.S. Nationalism and the 1970s ‘Bicentennial’ and ‘Americana’ Stamp Series,” Journal of Social History (forthcoming Fall 2015)

“The State Response to Occupy: Surveillance and Suppression,” in Todd A. Comer, ed., What Comes After Occupy? The Regional Politics of Resistance (forthcoming 2015).

“A White Man in the Colored Bronx,” in Teresa A. Booker, ed., Race and Urban Communities: An Interdisciplinary Approach (University of Akron Press, forthcoming 2015).

Saturday, July 5, 2014

Regarding FBI Official W. Mark Felt's (aka Deep Throat) Two Unpublished Manuscripts

In my recent book, Surveillance in America: Critical Analysis of the FBI, 1920 to the Present (2012), I devote a whole chapter to top FBI official W. Mark Felt, who we now know acted as Deep Throat during the Watergate crisis of 1972-1974. I detail how Felt led a secret faction at the FBI consisting of several other officials to undermine President Richard Nixon after Watergate and eventually to remove him from power.  The Felt faction worked with Washington Post reporters Bob Woodward and Carl Bernstein to break the Watergate story. However, if Felt had gone to Congress, instead of the press, to expose the Watergate break-in and associated crimes, the exposure would have occurred faster and probably would have led to Nixon losing the 1972 election.  In sum, Democratic candidate George McGovern would have been elected President.  By leaking slowly to the press, Felt helped Nixon win reelection.  
      Beyond that dramatic story, in my book I refer to two unpublished manuscripts written by Felt which have not seen the light of day.  In 1986, Felt submitted to the FBI for prepublication review and approval a book titled, An Unexpected Turn of Events.  In 1990, the FBI reviewed a second Felt manuscript entitled, Thirteen True Stories About the FBI.  The FBI reviewed both books and concluded that "no sensitive information" was revealed.  

      I have heard through the academic grapevine that scholars have made efforts to locate these unpublished manuscripts and have not been able to find them. These scholars have checked my sources in my Surveillance in America book to see how I know about them. 
      Information referring to these manuscripts are contained in the declassified FBI file on Felt.  I was the first person to request and obtain this FBI file under the Freedom of Information Act (FOIA) after Felt's death.  Soon after I received the Felt file, the FBI decided to put part of it on its own website (  Let me repeat, they put only part of it online. 

      I emphasis this fact because the FBI memos referring to Felt's two unpublished manuscripts are not contained in the Felt file that is posted on the FBI website.

      The FBI sent me a much larger file on Felt than they posted online -- about twice as large  I already have referred to this fact in a public lecture I gave in the summer of  2012 at the HOPE 9 conference in New York City.  My hour-long talk was video-taped and already has been posted on my blog.  Here is a link to the lecture ---
       The mystery of where Felt's two unpublished manuscripts are located never may be resolved.  But there should be no doubt that he wrote them and they went through the prepublication review process at the FBI.
Update -- A fellow researcher in FBI files tells me the FBI may retain copies in their files of both of Felt's unpublished manuscripts since they underwent prepublication review.  One strategy is to file FOIA record requests with the FBI on both manuscripts to see if they can be obtained through the declassification process.  

Monday, May 19, 2014

The "Greenberg" Movie and Me

For a couple of years, I have been reluctant to comment in print on Noah Baumbach’s movie, Greenberg (2010), which loosely is based on the relationship between my younger brother, Roger, and me.  As you may recall, the major characters in the film are Roger Greenberg, played by Ben Stiller, and Ivan Schrank, played by Rhys Ifans. The movie is set in Los Angeles in the contemporary period and follows their relationship as Roger recovers from a psychiatric condition and wants to do “nothing” and Ivan is uncomfortably struggling with issues of love and work.  The two characters, who once were close friends as musicians in a band, had not seen each other in a while. 
            Neither Roger nor Ivan in the film strictly is based on the real-life Roger or Ivan. But, Noah has created composite characters, drawing upon aspects of our personalities and life histories.  I have an older brother Michael in real-life and he, too, makes a cameo in the movie as the older brother of the Stiller character who has left country with his family on a vacation.  In fact, my older brother Michael often was absent from the real-life Greenberg family and had left the country for an extended period to study medicine overseas.  The real-life Michael never had close relationships with his two brothers, while Roger and me had a very close relationship for a long period.
            I have been reluctant to comment on the movie for a number of reasons.  First, I did not want to draw attention to myself or my family for strictly personal reasons.  It is highly unusual for a director to use the names of living people in a fictional film, and to base the movie in part on them.  Second, I did not want to interfere with Noah’s creation.  That is, I did not want to add a new layer of meaning to the film by exposing his characters, and making an association, with living people and their lives.  However, by now the movie is history, and no longer is a subject of conversation, so the time seems ripe to offer some comments on the ways the movie is true to real-life and also on the ways that the movie has impacted my life. I offer some “behind the scenes” details as well.  I hope Noah does not take offense at these comments, but it does not really matter since he never asked permission to use the Greenberg names in the film. Nor did he tell the Greenbergs about the film before it appeared.
            But I got wind of it ahead of time.  It was inadvertent, and something initially of a shock.  My girlfriend at the time, Nancy, had spent a Friday night at my apartment in the Bronx. In the morning, she went online on my computer to read the newspaper and several of her other favorite sites.  As a guilty pleasure, she occasionally would peruse the website of celebrity gossip columnist Perez Hilton.  She called it her “escape.”  Nancy is an American history professor, who studies immigration and ethnicity.  Her following of celebrity news is very far afield from her usual interests. 
            “Hey Ivan,” she says to me.  I am sitting in a lounge chair across from her in my bedroom.  “There is a movie coming out named Greenberg.
“What do you mean?” I respond.
“It is in production.  I am reading Perez Hilton,” she says sheepishly.
“What? What is it about?” I ask.
She says the director is Noah Baumbach.
I am surprised. “Noah Baumbach? I know him.”
Nancy begins to read the article out loud to me.  Perez reports the plans for the movie, which will star the actor Mark Ruffalo in the main role.
“There is a picture of Ruffalo,” she says. “He looks like you.”
I walk over to the computer and peek at the screen.  “Oh shit,” I say.  “There is an obvious resemblance.  I have seen some of Ruffalo’s movies.  I like the type of off-beat characters he often plays.
“Who else is in it?” I ask.
Nancy tells me, “They haven’t chosen a co-star yet.  It is set in L.A.”
“What’s the plot?” I ask. “Is Noah making a movie about me?” 
Nancy says, “There is a character named Ivan in the movie.”  She is in shock.  “Also Roger. Roger Greenberg.”  She knows that my younger brother is named Roger, although she has yet to meet him.
“Who plays Ivan?” I ask.  She tells me it is Rhys Ifans.  She relays his movie credits. Apparently, he has a big fan following, but I do not know his work well.
“Are you sure the name of the movie is Greenberg?
“Yes. Greenberg. One word.
“Oh my God,” I say.  “This is news to me.  I haven’t seen Noah in at least a decade.” I then begin to describe how I know Noah.  “You know, the name Ivan appeared in one of his other movies, as did the name Greenberg.  It was his film, The Squid and the Whale.”  Nancy, who I had been dating for about a year, is excited.  My emotions are mixed. What is happening?  What is the movie about?  What does the character Ivan say and do?  Roger and me?  I wonder if my brother knows about it.
            Noah’s parents and my parents used to be close friends while I was growing up during the 1970s.  My mother, Dolores, met Noah’s father, Jonathan, while they were undergraduates at Brooklyn College, CUNY, during the early 1950s.  At the time, my mother was married and my parents formed a lasting friendship with Jonathan that was solid for many years. Noah’s father and my parents earned doctorates and became college professors in the CUNY system.  Jonathan taught writing at Brooklyn College.  Dolores taught American history at Hunter College.  My father, Robert, taught English literature at Queens College.  They moved in the same intellectual circle in New York City.   When I was young, our families often got together socially, and the children became friends. Noah was several years younger than me and he was closer to my brother Roger.  Still, we had many dinners together and I must have impressed Noah in some way.  
            Over the years, Noah and I had very little contact, although Roger remained in touch with him.  Apparently, Noah heard details of my life from a close friend of my mother’s, Pat, who also remained close to Noah.  So my connection to Noah was made through Pat.  My mother used to tell me about Noah as she heard gossip from Pat.
            So, after I hear that the Greenberg movie is in production, I call both Roger and my mother to ask if they had heard about it.  No, they say. In fact, Roger, who had become a lawyer, was startled that his full name was being used.  He soon emailed Noah to inquire about the content of the film.  After Noah told him about the film, Roger reluctantly went along without making a fuss.  In fact, Roger, in a nod to me, asked Noah a favor.  My son, Andrew, was an undergraduate at UCLA not far from where the movie was being produced.  Could Noah let him visit the set or intern on the movie?  At first, Noah resisted this interference but eventually let Andrew come to the set and put him in the film as an extra, with some of his friends, in the long party scene at the end of the movie. Actually, Noah asked Andrew if he could round up ten of his friends to pose as party-goers in that scene.  He got paid $20 an hour and the long scene took a week to film.  My son was thrilled to be in the movie.
            When the movie premiered in New York, Noah asked Roger if he wanted to attend a special screening.  My brother turned down this invitation.  Why I'm not sure.  I have not spoken to Noah since the film came out.   
            My mother asked me if I liked the film and how I felt about my name and select details of my life being used.  I told her, “As long as the movie is first-rate, I don’t object. I don’t want to be associated with a mediocre film.  This is a first-rate movie.”  My reaction made its way back to Noah, probably via Pat, and I heard that Noah liked my response.
            In American culture, Hollywood movies often become bigger than real-life.  While Greenberg resembles an indie film more than a Hollywood film -- indie in content and sensibility, but Hollywood in its casting – its impact on my life has been curiously substantial.  I have been treated differently by friends, lovers, and work peers because of the film.  Now, I rarely discuss the film with people, but word of mouth gets around and, when I do discuss it, there often is that cliché of a reaction – “wow, you were in the movies…you are a star…are you more like the Ben Stiller or Rhys Ifans character?, someone made a movie about you.” 
            In real-life, I earned a doctorate in American history, taught college for a decade, and recently wrote two nonfiction books on civil liberties in America. Among my professor-type work peers, I have tried to hide any association with the film.  High-minded scholars probably would frown on such a film, and misunderstand me, since no character in Greenberg is an intellectual. Greenberg is not a film about, or for, intellectuals, although it is a very smart movie.  Noah already addressed in a critical way the intellectual world of writers and professors in The Squid and the Whale, which in large measure is a critique of his parents.  However, among my friends I have mentioned Greenberg and was interested in their reaction to the film.  I heard diverse responses.  Some friends liked the film and could not believe the movie explored the relationship between my brother and me.   Again, it is the Hollywood syndrome. “Wow, you are mentioned in a movie – how cool is that.”  Some friends were competitive or jealous and voiced negative reactions.  They did not like the film and wondered if I was angry that Noah had depicted Stiller as a former “mental patient.”  Another common reaction was for people to notice that my personality in real-life was a little bit of both Stiller and Ifans in the film – how could it be both?  I would explain these are composite characters and if someone asked Noah about them, he probably would deny they resembled either me or Roger in real-life.  There have been times when I acted in some way and a person says -- “Yeah, that is exactly what Stiller would have done.”  People compare me to the fictional creation.  They seem to see me as closer to the Stiller character because of Stiller’s practice of writing angry complaint letters and his anti-corporate attitude.           
            Noah's focus on my relationship with my brother rings true in some ways. For many years we were close – Roger and I used to drink together, go to Knicks basketball games together, and socialize at times with the same people.  That was when we were much younger.  Today, Roger and I have had a serious falling out – in much the same way the fictional characters have a falling out at the end of the movie.  But, my break with Roger came after the movie and was unrelated to it.  The tension between us got so bad that at one point that Roger even told his two young children that I had died.  What an extreme response.  When I told a close friend what Roger had done -- and said that I had passed -- she commented, “Oh, that is what Ben Stiller would have done.”  I laughed.