we have a signal recording and there are always interventions, I think it is not a noise, since it is not continuous but occurs in intervals, with 25 Hz (please see below). It is not affected by the sampling rate, and the internal frequency is near 1000 Hz. I used median filter, Butterworth filter, and moving averages filter to filter it out. I am to a degree successful but when we increase the degree of the filter we lose our data which is important for us. I use Octave for data evaluation. Has anybody had a similar experience? Or how can I remove the intervention without damaging the original signal? Thanks a lot in advance.
If I understand your plot correctly, the peaks appear at and around multiples of about 100 Hz in your spectrum. (Is this a Hz scale?) That sounds suspiciously like harmonics of double the mains frequency (at least in Europe and other parts of the world). Are you sure this isn’t caused by some of you lab equipment?
I can’t comment on filtering that “noise” signal out efficiently without affecting the “actual” signal. You probably already looked into avoiding it in the first place. But that would be the “ideal” way to move forward imho.
What sample rate is used for this graph?
I would sample it at about 8 times the frequency that is in the “noise”. Then do a lowpass, and then downsample to a reasonable sample rate.
This is just a guess from far away.
As was said , it would be much better to find the source of the noise and eliminate it. It could be a switching power supply that is at a low usage rate and is pulsing on and off at the line frequency.
It is not a Hz scale. It shows the number of data. The sampling rate is 2kHz and the frequency of these events is approximately 20 Hz. As it can be seen it is the in each 100 data. We suspect it is coming from the wireless unit but anyway we need it. I must set a 50 Hz filter to eliminate completely then we lose our original signal. That is the problem. Thanks a lo for the comment.
The sampling rate is 2KHz and a lower rate is not an option for us. The frequency of these events is 20 Hz (I wrote 25 Hz but it is 20 Hz). Thanks for replying.
Ok. The frequency of the event is 20Hz. but what is the frequency inside the event? I think you should sample at 44KHz. and then look at it closely.
Inside the event it is near 1kHz, I mean the frequency of individual peaks of the interruptive signal.
If you are only sampling at 2Kz then you will not see anything above 1Kz.
please sample higher.
Ye, I know. It is clear but we have now the capacity to measure with up to 2kHz and we don’t need higher. 1kHz is from the false signal not our measurements.
How do you know that the false signal is 1 KHz. Do you understand aliasing?
If you can’t sample higher than 2KHz. then you need to do an analogue filter before you sample.
When there is no measurement signal, there is only the false signal. Then I can calculate the frequency of an individual event, it is not tricky.
Is the value of the useful signal always inside some bound ? I can see that the signal always appears to be below 2.5 in your plot. If you are really sure that useful signal is always within some bound
[a, b], then you can try to replace such data points with average of the adjacent samples.
To find such out of range data points you can do
logical_index_of_out_of_range_elements = (signal < a) | (signal > b); numerical_index_of_out_of_range_elements = find(logical_index_of_out_of_range_elements);
Another property of the intervention signal appears that they are very different in magnitude from adjacent samples. If you are sure that useful signal always have adjacent sample difference in the range
[-a to a], the you can find out of range data points by doing
diff_of_adjacent_samples = diff(signal); % the result has one element fewer than the input. diff_of_adjacent_samples = abs(diff_of_adjacent samples);
The first one seems reasonable but find function returns #VAL instead of the indices.
I converted the logical to double but it did not work.
This is what I get
xx = rand(1,20) xx = Columns 1 through 8: 0.364622 0.185030 0.835353 0.031313 0.968865 ... >> yy = (xx < 0.1) | (xx > 0.9) yy = 0 0 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 >> find(yy) ans = 4 5 9 14
This worked also for me. I will check again mine.