Data analysis of a file of bladeRF samples with python

Finally, some python-programming

With bladeRF-CLI, the bladeRF-control-program, one can collect received data into a file. That file contains I- and Q-samples. With a very simple python-program I try to get some insight in the received data.

First python-program for bladeRF-datafile

Create a data-file with bladeRF-cli:

bladeRF> version
[WARNING] FPGA currently does not have a version number.

Serial #:               b436de8c8212b9aeaaeba852246866e7
VCTCXO DAC calibration: 0x9a5d
FPGA size:              115 KLE
Firmware version:       1.5
FPGA version:           0.0

bladeRF> print bandwidth

  RX Bandwidth:  28000000Hz
  TX Bandwidth:  28000000Hz

bladeRF> print samplerate
[INFO] Calculated samplerate: 1000000 + 0/1
[INFO] Calculated samplerate: 1000000 + 0/1
  RX sample rate: 1000000
  TX sample rate: 1000000
bladeRF> set frequency 800000000

  Set RX frequency:  800000000Hz
  Set TX frequency:  800000000Hz

bladeRF> rx config format=csv n=1024 file=C:/temp/A800MHzCW.csv
bladeRF> rx start
bladeRF>

And now analyse the datafile with a python-program

f = open('/Temp/A800MHzCW.csv', 'r')
n = 0
averageI = 0
averageQ = 0
for line in f:
    if n < 10:
        print line,
    list = line.split(',')
    I = int(list[0])
    Q = int(list[1])
    #print "I =",I, "Q =",Q
    averageI = averageI + I
    averageQ = averageQ + Q
    n = n + 1
averageI = averageI / n
averageQ = averageQ / n
print f
print "averageI = ", averageI, " averageQ = ", averageQ

and the output is:

Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>> 
110, 100
113, 100
112, 103
114, 104
112, 105
110, 101
114, 100
112, 100
115, 97
113, 97

averageI =  110  averageQ =  98
>>>

This may seem a silly simple program but:

• I can read a file
• I can interpret the content and get I and Q samples
• I can print values
• I can calculate some data

So, this is a sort of 'Hello world' program.

Plotting in python

How?
I need to install matplotlib for python 2.7
Download from http://sourceforge.net/projects/matplotlib/postdownload?source=dlp
And execute matplotlib-1.3.1.win-amd64-py2.7.exe

Helloo, I _have_ python 2.7 installed!!!

I have python 2.7 running on my machine, obviously not registered properly?
I’’ try matplotlib-1.3.1.win32-py2.7.exe instead

Seems to be ok.

But...

So...

From http://www.lfd.uci.edu/~gohlke/pythonlibs/
python-dateutil-1.5.win32-py2.7.exe

>>> import matplotlib

Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    import matplotlib
  File "C:\Python27\lib\site-packages\matplotlib\__init__.py", line 125, in <module>
    raise ImportError("matplotlib requires pyparsing")
ImportError: matplotlib requires pyparsing
>>>

From http://www.lfd.uci.edu/~gohlke/pythonlibs/
pyparsing-2.0.1.win32-py2.7.exe

>>> import matplotlib
>>>

Quite encouraging!

All examples from internet about plotting with matplotlib function very well!

Nice playing by the way!!

Sometimes it is difficult to go persistently only into the direction of your goal. Before my retirement I had to focus on result, that was very important. But there are so many distracting sidepaths... Nowadays I play a lot, go into an enormous lot of sidepaths, and that is great fun! So, progress is not the issue anymore...

Some data-files

summary:
freq = 1575.450.000 Hz
bandwidth = 28.000.000 Hz
samplerate = 1.000.000 Hz
file=/temp/AA.csv
n = 5000 samples

# FFTSDRdata01.py
# Oct-2013 Kees de Groot
#
# show data collected from bladeRF
import pylab as pl
import numpy as np
import math

filename = 'AA.csv'
f = open('/Temp/' + filename, 'r')
print f
n = 0
dataI = []
dataQ = []
dataC = []

for line in f:      # print 1st 10 lines of data-file
    if n < 10:
        print line,
    list = line.split(',')
    I = int(list[0])
    Q = int(list[1])
    dataI.append(I)
    dataQ.append(Q)
    dataC.append(complex(I,Q))
    n += 1
    
print "n = ", n

pl.figure(1)
pl.title("Q versus I of " + filename)
pl.xlabel('=======>  I')
pl.ylabel('=======>  Q')
pl.plot(dataI, dataQ, 'ro')

pl.figure(2)

pl.subplot(211)
pl.title(filename)
pl.ylabel('=======>  I')
pl.plot(dataI)
pl.title("I-values of " + filename)

pl.subplot(212)
pl.title(filename)
pl.ylabel('=======>  Q')
pl.plot(dataQ)

pl.figure(3)

fsample = 1E6
timestep = 1 / fsample

sp = np.fft.fft(dataC)
freq = np.fft.fftfreq(n, d=timestep)
pl.title(filename)
pl.plot(freq, abs(sp))
pl.show()

Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>>
<open file '/Temp/AA.csv', mode 'r' at 0x0343D390>
80, 80
82, 77
81, 77
77, 79
80, 77
80, 81
78, 79
81, 76
84, 80
80, 76
n =  5000

No rocket-science, but quite encouraging.

I see some interruptions/setting-changes/discontinuities, but what-the-heck: nice pictures!

(I may be a strange person, but I really became quite excited when I first got these pictures!)

Settable gains inside the bladeRF-board

From the log of SDR-Radio console:

16:33:51> bladeRF

16:33:51>     --Starting sending

16:33:51>     Set RX VGA gain 2: value 30dB, status = Success

16:33:51>     Set RX LPF mode: Normal, status = Success

16:33:51>     Set RX VGA gain 1: value 30dB, status = Success

16:33:51>     Set LNA gain: Maximum, status = Success

16:33:51> Input Reader (Part 1)

16:33:51>     Starting listener, Radio defn 04A686B0, buffer size 65536

There is a third settable gain: lna-gain.

[INFO] Using libusb version 1.0.16.10774
[INFO] Found a bladeRF
[INFO] Claimed all inferfaces successfully
[INFO] Change to alternate interface 1
[INFO] Changed into RF link mode: LIBUSB_SUCCESS / LIBUSB_TRANSFER_COMPLETED
[WARNING] extract_field: Field checksum mismatch
[WARNING] Could not extract VCTCXO trim value
[WARNING] extract_field: Field checksum mismatch
[WARNING] Could not extract FPGA size
bladeRF> print bandwidth

  RX Bandwidth:  28000000Hz
  TX Bandwidth:  28000000Hz

bladeRF> print frequency

  RX Frequency: 1575450000Hz
  TX Frequency: 1575450000Hz

bladeRF> print samplerate
[INFO] Calculated samplerate: 38400000 + 0/1
[INFO] Calculated samplerate: 1000000 + 0/1
  RX sample rate: 38400000
  TX sample rate: 1000000
bladeRF> print vga1gain

print: Invalid parameter (vga1gain)

bladeRF> print gainvga1

print: Invalid parameter (gainvga1)

bladeRF> help set

set  

The set command takes a parameter and an arbitrary number of
arguments for that particular command.  The parameter is one
of:

   bandwidth       Bandwidth settings
   config          Overview of everything
   frequency       Frequency settings
   lmsregs         LMS6002D register dump
   loopback        Loopback settings
   mimo            MIMO settings
   pa              PA settings
   pps             PPS settings
   refclk          Reference clock settings
   rxvga1          Gain setting of RXVGA1 in dB (range: )
   rxvga2          Gain setting of RXVGA2 in dB (range: )
   samplerate      Samplerate settings
   trimdac         VCTCXO Trim DAC settings
   txvga1          Gain setting of TXVGA1 in dB (range: )
   txvga2          Gain setting of TXVGA2 in dB (range: )

bladeRF> print rxvga1

  RXVGA1 Gain:  30

bladeRF> print rxvga2

  RXVGA2 Gain:  30dB

bladeRF>






nope, no other gain-setting.

What is the maximum data-rate / samplerate for USB2?

A lot of different experiments lead to the following python-program:
A.csv is a file without any connection from bladeRF to the signal generator, so, pure noise


The python-program:

# FFTSDRdata03.py
# Nov-2013 Kees de Groot
#
# show and analyse snapshot file from bladeRF
# remove DC-component
# show data in different ways; I/Q, realtimeplot, FFT

import pylab as pl
import numpy as np
import math

filename = 'A.csv'
fsample = 1E6
timestep = 1 / fsample

f = open('/Temp/' + filename, 'r')
print f
print "fsample = ", fsample

dataI = []
dataQ = []

n = 0
sumI = 0
sumQ = 0
# read I, Q - values into memory
for line in f:
    list = line.split(',') # two values with comma inbetween
    Q = int(list[0])
    I = int(list[1])
    # print 1st 10 lines of data-file
    if n == 0:
        print '1st 10 lines of I/Q-values'
    if n < 10:
        print I, Q
    dataI.append(I) # save data in memory
    dataQ.append(Q)    
    sumI += I
    sumQ += Q
    n += 1
    
averI = sumI / n  # calculate average
averQ = sumQ / n

print "n = ", n

# remove DC-component and construct complex dataC
dataC = []
for i in range(n):
    I = dataI[i] - averI
    Q = dataQ[i] - averQ
    dataI[i] = I
    dataQ[i] = Q
    dataC.append(complex(I,Q))  # this one has complex data
    
pl.figure(1)
pl.title("Q versus I of " + filename)
pl.xlabel('=======>  I')
pl.ylabel('=======>  Q')
pl.plot(dataI, dataQ, 'ro')

pl.figure(2)
pl.subplot(211)
pl.title("I-values of " + filename)
pl.ylabel('=======>  I')
pl.plot(dataI)
pl.subplot(212)
pl.title("Q-values of " + filename)
pl.ylabel('=======>  Q')
pl.plot(dataQ)

pl.figure(3)
sp = np.fft.fft(dataC)      # use fft, not rfft, since dataC is complex
freq = np.fft.fftfreq(n, d=timestep)    # for convenient x-axis-ticks
pl.title('fft of ' + filename)
pl.xlabel('frequency')
pl.ylabel('magnitude')
pl.plot(freq, abs(sp)/n)    # abs converts to magnitude; /n normalizes 

pl.show()

With output
A.csv is a file without any connection from bladeRF to the signal generator, so, pure noise:

There is an enormous peak at 200kHz which was not there before on other pictures. I have to find out what is happening there.

I did a lot of more measurements:

BladeRF is setup:
RX Bandwidth:  28.000.000Hz
RX sample rate: 1.000.000
RX frequency:  646.000.000 Hz
RXVGA1 Gain:  33
RXVGA2 Gain:   3dB
A.csv is a file without any connection from bladeRF to the signal generator, so, pure noise
Now the signal generator, a HP8656B is directly connected to RX of bladeRF
B.csv is 646 MHz, AM 1 kHz 75% modulation, 1 µV amplitude
C.csv is 3,98 µV
D.csv is 7.94 µV
E.csv is 31.6 µV
F.csv is 63,1 µV

Back to 1 µV and now FM 1 kHz sweep 75 kHz wide

AF.csv is 1 µV
BF.csv is 3.98 µV
CF.csv is 7.94 µV
DF.csv is 31,6 µV

This last signal, DF.csv, is very nice:

Conclusion

I managed to get a file with data (I and Q-samples) from bladeRF-board. With a program written in python I managed to read these files, interpret and show the data. Nice results, and a big question: what is that signal at 200kHz? So, every experiment gives you new things to investigate...

Wife: what are you doing?
Me: Well, you know, my project..
Wife: What are you trying to achieve?
Me: Well, at the end this board will tell me where I am..
Wife: Do you know where you are at the end?
Me: Yes..
Wife: So, ???

This whole project, that keeps me out of all evil places of the world, that keeps me home, ok, that keeps me behind my laptop the whole evening and the first part of the night, that keeps me active,
h o w  c a n  I  e v e r  e x p l a i n  t o  a n  a l i e n  w h a t  I  a m  d o i n g  h e r e?
My wife stops asking when I speak the magic spell: "it is just my hobby!"